AD-A268  132 


llilllllllillllilllllil 


PROCEEDINGS 


s 


DTIC 

ELECT-  ^ 
AUGl  6  19931 


c 


1993  ACQUISITION 
RESEARCH  SYMPOSIUM 


ACQUISITION  FOR 
THE  FUTURE: 

IMAGINATION,  INNOVATION, 
AND  IMPLEMENTATION 


RESEARCH  AND  REALITY; 
CLOSING  THE  GAP 


kpptor*c  :o-  r*^«  j 

DiftaDtitiac  _ _ _ 


93-- 187 10 


CO-HOSTED  BY  THE 

DEFENSE  SYSTEMS  MANAGEMENT  COLLEGE 
AND  THE  NATIONAL  CONTRACT  MANAGEMENT  ASSOCIATION 
WASHINGTON,  DC  CHAPTER 


TfflS  DOCUMENT  IS  BEST 
QUALITY  AVAILABLE.  THE  COPY 
FURNISHED  TO  DWC  CONTADIED 
A  SIGNIFICANT  NUMBER  OF 
PAGES  WHICH  DO  NOT 
REPRODUCE  LEGIBLY. 


1993  ACQUISITION  RESEARCH  SYMPOSIUM 


Acquisition  for  the  Future 


Imagination,  Innovation,  and  Implementation 
Research  and  Reality:  Closing  the  Gap 


SYMPOSIUM  HOSTS 

Defense  Systems  Washington  D.C  Chapter, 

Management  College  National  Contract 

Management  Association 


DSMC  CONFERENCE  OFFICIALS 


DTIC  3 

Mr.  Calvin  Brown:  Conference  Co-Chair 


Ms.  Joan  Sable:  Program  Co-Chair 


NCMA  CONFERENCE  OFFICIALS 


Ms.  Donna  Ireton:  Conference  Co-Chair 
Mr.  Patrick  Sullivan:  Program  Co-Chair 
Ms.  Dina  Hyde:  Arrangements  Chair 
Dr.  Alma  Davis:  Moderator  Chair 
Ms.  Shirley  Bass:  Registration  Chair 


hAccesion  For 

,  ntis  CRA&I 
j  DflC  tab 

(  I' oj  ;crcl 


Jll  jtltlCdfi  ■ 


Uist'ibi'tion  I 


Codes 


Oist 


Av/dii  c»''dt*r 
Soecial 


The  1993  Acquisition  Research  Symposium  is  the  latest  in  a  series  of  conferences 
begun  in  1972.  These  Symposia  offer  a  dynamic  forum  for  dialogue  among  key  professionals 
working  on  vital  issues  facing  the  acquisition  community.  Attendees  include  senior  officials, 
program  managers,  staff  officers,  and  researchers  from  the  Department  of  Defense,  federal 
civilian  agencies,  academia,  and  industry. 

This  year’s  theme  and  subtheme  reflect  the  future  innovation  and  implementation  in 
the  acquisition  process.  "Acquisition  for  the  Future  -  Imagination.  Innovation,  and 
Implementation"  and  "Research  and  Reality:  Closing  the  Gap"  are  the  prevailing  theme 
and  subtheme  discussed  and  examined  throughout  this  publication.  The  papers  included 
cover  the  latest  research  and  development  as  documented  by  individuals  involved  in  the  many 
aspects  of  the  acquisition  process. 

Vie  invite  you  to  take  advantage  of  this  publication,  which  expands  upon  Symposium 
presentations  and  introduces  new  authors  and  topics.  Please  note  that  the  views  expressed 
are  those  of  the  authors  and  do  not  necessarily  reflect  the  views  of  the  organization  with 
which  they  are  associated. 
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ABSTRACT 

Financial  resources  for 
acquisition  policy  research  are 
largely  absorbed  in  paying 
salaries  to  people  who  do  the 
research.  Vast  resources  — 
resources  that  do  not  arise  from 
Department  of  Defense  (DoD) 
funding  for  acquisition  research 
organizations  —  are  currently 
available  to  the  DoD  for  defense 
acquisition  policy  studies 
through  U.S.  universities. 

Fiscal  constraints  and  public- 
policy  imperatives  may  require 
the  Dob  to  rely  more  heavily  on 
the  resources  of  the  nation's 
universities  for  acquisition 
policy  research  than  it  has  in 
the  past. 


BACKGROUND  AND  THE  PROBLEM 

In  1989  the  Defense  Systems 
Management  College  started  a 
research  project  called  Research 
on  Ongoing  Acquisition  Research 
(ROAR) .  The  project  sought  to 
discover  the  types  and  extent  of 
ongoing  research  activities 
conducted  by  professionals 
inside  and  outside  the  DoD  on 
defense-related  acquisition 
policy  issues. [1]  The  ultimate 
purpose  of  ROAR  was  to  create  an 
electronic  information  system  to 
satisfy  two  objectives. 

The  first  objective  was  to 
enable  acquisition  researchers 
to  access  ROAR  information 
freely  and  identify  other 
researchers  who  were  working 
simultaneously  on  similar 


acquisition  issues.  This  would 
help  them  to  accelerate  their 
studies  through  mutually 
benefical  exchanges  of  empirical 
data  and  ideas. 

Second,  DoD  policymakers  could 
use  ROAR  information  to  review 
ongoing  acquisition  studies  on 
topics  of  immediate  policy 
interest.  Thus  they  might  avert 
expenditures  for  research  start¬ 
ups  that  would  duplicate  work 
already  in  progress. 

Additionally,  policymakers  might 
quickly  obtain  useful  findings 
fj  jm  projects  that  were  nearing 
c  mpletion  but  were  not  yet 
published  or  otherwise 
identifiable  through  a  search  of 
databases  centrally  maintained 
by  the  Defense  Technical 
Information  Center  (DTIC) , 
Defense  Logistics  Studies 
Information  Exchange  (DLSIE) , 
CompuServe,  government 
libraries,  etc. 

A  discussion  of  ROAR'S  current 
status  appears  in  Appendix  1. 

ROAR  began  with  the  aim  of 
developing  abstracts  of  ongoing 
studies  being  performed  within 
all  relevant  research 
communities.  For  example,  it 
was  known  that  thousands  of 
reports  related  to  acquisition 
policy  are  funded  each  year  by 
DoD.  The  research  underlying 
the  reports  is  conducted  by  the 
DoD's  own  employees,  by  the 
employees  of  its  Federally 
Funded  Research  and  Development 
Centers  (FFRDCs) ,  and  by  other 
contractors . [ 2 ] 
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A  review  of  DoD's  centralized 
bibliographic  databases  yielded 
huge  lists  of  published 
acquisition  studies  prepared  by 
researchers  within  DoD.  Within- 
DoD  research  is  defined  here  to 
include  studies  performed  by  the 
DoD's  employees,  FFRDCs,  and 
for-profit  contractors.  The 
same  databases  offered  little 
evidence  that  similar  research 
was  performed  by  other  groups  — 
by  professors  in  universities 
for  example  —  with  DoD 
financing  or  otherwise. [3 ] 

However,  an  informal  review  at 
that  time  of  articles  in 
academic  and  professional 
journals  for  selected 
disciplines  (e.g.,  engineering, 
economics,  law,  political 
science,  operations  research, 
finance,  business  and 
management,  etc.)  suggested  that 
university  scholars  had 
conducted  defense-related 
acquisition  policy  studies  as 
well . 

To  ensure  ROAR's  coverage  of 
acquisition  studies  would  not 
overlook  possible  major 
producers  of  such  work,  the 
decision  was  made  to  pursue 
information  about  ongoing 
research  projects  at 
universities  and  other 
organizations  outside  DoD  (e.g., 
non-profits  without  affiliation 
with  the  DoD  and  trade 
associations)  as  vigorously  as 
within-DoD  project  information. 

The  source  of  funding  for 
within-DoD  research  was  evident; 
the  Defense  Department  provided 
most  or  all  of  the  funds.  But 
if  many  ongoing  projects  in 
defense  acquisition  policy  were 
found  to  exist  at  universities, 
it  would  be  interesting  to 
determine  their  funding  sources. 
Three  different  sources  of 


funding  for  such  projects  seemed 
possible. 

First,  it  was  possible  that  the 
DoD  provided  direct  funding  for 
many  university  efforts.  But  in 
this  case,  the  mechanisms  for 
including  descriptions  of  the 
research  in  the  DoD's 
centralized  bibliographic 
databases  were  evidentally 
imperfect. 

The  second  possibility  was  that 
private  corporations,  non-DoD 
federal  agencies,  and 
foundations  provided 
universities  with  direct  funding 
for  some  university  research. 
In  this  case,  the  DoD's 
centralized  databases  would  not 
normally  capture  information 
about  these  studies. 

The  third  possibility  was  that 
universities  effectively  funded 
some  of  the  research  themselves. 
Professors  are  normally  paid  by 
their  university  to  conduct 
research  of  their  choosing  as 
well  as  to  teach.  It  would  be 
understandable  for  professors  in 
many  disciplines  to  undertake 
original  research  on  issues  in 
defense-related  acquisition 
policy  to  demonstrate  their 
disciplines'  capacity  to 
illuminate  key  public-policy 
issues  and  options. 


HYPOTHESIS 

It  was  hypothesized  that  at 
least  25%  of  all  university 
projects  discovered  by  ROAR 
would  prove  to  be  funded 
directly  by  the  Department  of 
Defense.  In  developing  the 
hypothesis,  it  was  assumed  that 
the  particular  method  chosen  to 
elicit  ongoing  research 
descriptions,  including 
sponsors'  identities,  would  lead 
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naturally  to  a  high  incidence  of 
DoD  sponsorship  reported  in  the 
research  performed  in 
universities.  The  true 
incidence  was  hypothesized  to  be 
somewhat  less  than  25%  in  the 
population,  but  it  was  believed 
that  an  unbiased  estimate  would 
not  be  obtainable  using  the 
elicitation  method. 

The  elicitation  method  involved 
chaining  through  telephone 
interviews  with  direct 
referrals.  Initially,  five  DoD 
researchers  would  be  contacted 
individually.  They  would  be 
asked  to  describe  their  ongoing 
research,  if  any,  dealing  with 
defense  acquisition  policy.  At 
the  conclusion  of  the  interview, 
they  would  be  asked  to  identify 
anyone  else  currently  conducting 
defense  acquisition  research, 
regardless  of  whether  it  was 
related  to  their  work  or  not. 
The  referrals  would  then  be 
contacted,  interviews  would  be 
repeated,  new  referrals  would  be 
sought,  and  so  on. 

It  was  assumed  that  within-DoD 
researchers  would  tend  to  be 
aware  of  other  projects  in 
progress  that  were  financed  by 
the  DoD  and  less  aware  of  those 
not  financed  by  the  DoD.  The 
reason:  the  easiest  way  to  learn 
about  ongoing  work  prior  to  its 
publication  is  through  direct 
personal  contacts;  and  such 
contacts  occur  more  often 
between  a  within-DoD  analyst  and 
his  or  her  co-workers  employed 
in  the  same  facility  than 
between  a  within-DoD  analyst  and 
university  scholars. 

It  was  further  assumed  that, 
when  an  interview  with  a  DoD 
researcher  did  produce  a 
referral  to  a  university  scholar 
conducting  acquisition  research, 
the  referral  would  tend  to  be 


based  on  the  fact  that  the 
university  effort  was  funded  or 
otherwise  sponsored  by  the  DoD 
organization  from  which  the 
referral  came.  At  least, 
insofar  as  the  organization 
actively  funded /sponsored  any 
projects  in  universities,  this 
was  thought  to  be  a  likely 
outcome . 

In  short,  it  was  assumed  that 
chained  interviews  to  elicit 
descriptions  of  ongoing 
acquisition  policy  research, 
commencing  with  within-DoD 
researchers  and  their  personal 
referrals,  would  bias  upward  the 
reported  incidence  of  DoD 
funding  for  the  university 
research  projects  that  were 
discovered.  Such  a  bias  would 
lead  to  the  discovery  of  at 
least  one  university  project 
directly  funded/ sponsored  by  DoD 
for  every  three  reported  to  be 
funded  in  some  other  way.  This 
was  the  hypothesis  tested  by  the 
ROAR  project. 


FINDINGS  AND  DISCUSSION 

Interviews  with  researchers 
began  in  December  1989  and 
continue  today.  In  November 

1991  a  preliminary  analysis  was 
done  based  on  interview  data  for 
ongoing  acquisition  policy 
studies  conducted  by  university 
scholars.  Those  data  showed 
that  5%  of  university  projects 
had  been  financed  or  otherwise 
sponsored  by  DoD . [ 4 ]  Subsequent 
analysis  of  data  for  ongoing 
studies  collected  as  of  November 

1992  yielded  the  same  5%  figure. 

The  data  shown  below  summarize 
the  results  of  the  1992 
analysis.  Because  the  finding 
of  5%  incidence  is  unchanged 
from  the  1991  analysis,  this 
paper  presents  results  based  on 
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the  more  recent  data,  as  of  late 
1992. 


Ongoing  Projects  in  ROAR 
Univ  Proi  All  Proi 


Total  Number:  370  876 

Total  funded 
/ supported 

by  DoD:  19  504 


These  data  show  that  5.1%  (19  of 
370)  university  projects,  or 
about  one  in  twenty  projects 
discovered  through  chained 
interviews,  were  directly 
funded/ supported  by  the  Defense 
Department.  The  5.1%  estimate 
for  the  sample  proportion  is 
significantly  different  from  the 
hypothesized  25%  proportion  at 
the  0.01  level. 

It  is  possible  that  some 
researchers  simply  avoided 
reporting  DoD  support  for  their 
work  for  unknown  reasons. 
However,  each  was  told  that  DoD 
policymakers  would  have  access 
to  their  descriptions.  Thus  it 
is  unlikely  that  many  would  risk 
having  their  DoD  sponsors  learn 
they  had  misrepresented  the  fact 
of  their  sponsorship. 

Among  the  19  DoD-sponsored 
studies,  the  most  frequently 
cited  sponsor  was  the  Office  of 
Program  Analysis  and  Evaluation 
(PA&E) ,  within  the  Office  of  the 
Secretary  of  Defense.  In  the 
late  1980s,  PA&E  initiated 
support  for  university  research 
in  economics  and  other 
disciplines  to  address  defense 
acquisition-related  issues.  The 
RAND  Corporation  administered 


the  projects  on  behalf  of 
PA&E. [5]  Other  university 
studies  funded  by  DoD  reported 
various  Army,  Navy  and  Air  Force 
sponsors. 

The  1992  analysis  showed  that  a 
total  of  89  of  the  370 
university  projects,  or  more 
than  a  fifth,  enjoyed  external 
sponsorship.  In  addition  to  the 
19  DoD-sponsored  efforts  already 
noted,  70  projects  were 
variously  supported  by  private 
foundations  (e.g..  Pew 
Charitable  Trust,  Bradley 
Foundation,  01  in  Foundation, 
Twentieth  Century  Fund) , 
corporations  (e.g.,  Unisys,  IBM, 
Hughes,  Boeing,  General 
Electric) ,  non-DoD  federal 
sources  (e.g..  National  Science 
Foundation,  Department  of 
Commerce,  NASA,  FAA)  ,  and  a  wide 
range  of  professional  and  trade 
associations. 

It  is  noteworthy  that  university 
research  represented  more  than 
40%  of  all  defense  acquisition 
policy  studies  discovered  by 
ROAR  interviews.  And  more  than 
three-quarters  of  the  university 
projects  (281  out  of  370)  were 
apparently  begun  at  the 
initiative  of  professors  and 
fellows  with  direct  funding  from 
their  universities. 

Another  significant  result  of 
the  data-collection  effort 
concerns  the  observed  pattern  of 
referrals  through  interviews. 
Although  statistics  were  not 
maintained  to  characterize  the 
pattern,  a  finding  based  on  non- 
empirical  observation  helps  to 
explain  why  the  hypothesized 
incidence  of  25%  for  DoD 
sponsorship  of  university 
projects  was  not  confirmed  with 
actual  data.  This  finding  also 
may  hold  important  policy 
implications  for  future  defense 
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acquisition  research. 


Bridging  Separate  Communities. 

The  previously  stated  assumption 
that  within-DoD  researchers 
would  tend  to  offer  referrals  to 
other  DoD  researchers  was 
confirmed.  Of  course,  DoD 
researchers  occasionally 
volunteered  the  names  of 
university  researchers  as  well. 

When  these  university 
researchers  were  contacted,  they 
rarely  indicated  knowledge  of 
ongoing  activities  within  the 
DoD  research  community  (although 
many  said  they  were  interested 
in  learning  more)  .  In  turn, 
their  referrals  tended  to 
include  other  university 
researchers  with  whom  they  were 
familiar. 

By  mid-1990,  the  interview 
process  had  divided  with 
unexpected  clarity  into  two 
tracks.  One  track  produced 
chained  interviews  about  within- 
DoD  research  underway,  with 
referrals  to  like  researchers. 
The  other  track  generated 
interviews  with  university 
scholars  about  their  studies, 
with  referrals  mainly  to  other 
university  researchers. 

ROAR'S  interviewers  found  they 
could  provide  a  valuable 
informal  service,  where 
possible,  by  alerting  one 
researcher  to  the  efforts  of 
another  researcher  who  had  been 
interviewed  previously  —  based 
on  the  interviewer's 
recollection  that  the  two  were 
studying  a  common  issue.  But  as 
the  aggregation  of  projects  rose 
to  many  hundreds,  it  became 
difficult  to  continue  the 
service  with  human  interviewers 
who,  lacking  eidetic  memories. 


could  not  instantly  recall  the 
details  of  every  prior  ROAR 
interview.  However,  the 
revealed  need  for  the  cross¬ 
dissemination  of  pertinent 
ongoing-research  descriptions 
was  specifically  addressed  in 
developing  ROAR'S  electronic 
information  system. 

If  the  incidence  of  DoD- 
sponsored  university  research 
had  been  estimated  in  mid-1990 
(with  six  months  of  data) ,  it  is 
possible  that  a  figure  higher 
than  5%  would  have  resulted. 
However,  it  appears  that  the 
collection  of  data  along  two 
tracks  simultaneously  for  a 
period  of  years  obscured  the 
effect  of  choosing  only  within- 
DoD  researchers  to  commence  the 
interviews . 

The  collection  of  data  by 
interviews  continues  today  on 
both  tracks.  The  rate  of 
progress  indicates  that  the 
projects  already  reported  to 
ROAR  represent  small  fractions 
of  all  ongoing  studies  in  DoD- 
funded  organizations  and  in  the 
university  research  community. 

The  1992  findings  support  the 
view  that  universities  are 
conducting  a  large  number  of 
research  projects  on  defense 
acquisition  policy;  and  few  of 
the  projects  appear  to  rely 
directly  on  Defense  Department 
resources. [6] 


Additional  Findings:  Types  of 
University  Projects. 

It  may  be  useful  to  know  how 
university  acquisition  research 
projects  in  progress  vary  in 
broad  type  from  all  other 
projects  (mainly  within-DoD 
projects)  captured  by  ROAR.  To 
observe  differences,  twelve 
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categories  of  defense 
acquisition  policy  research  were 
established. 

All  876  ROAR  projects  were 
classified  by  broad  acquisition 
category.  The  classification 
procedure  allowed  each  project 
to  be  classified  as  belonging  to 
at  least  one  and  no  more  than 
four  categories,  depending  on 
the  relevance  of  the  research  to 
the  respective  categories.  This 
procedure  recognized  that  a 
given  study,  say  a  project 
concerning  new  defense 
contracting  rules  for  acquiring 
commercial  items,  would  be 
properly  classified  as  relevant 
to  two  of  the  categories: 
Commercial  Products  &  Practices; 
and  Contracting  &  Subcontracting 
Issues.  Rarely  was  a  project 
found  to  "belong'*  to  as  many  as 
four  categories. 

The  results  of  the 
classification  procedure  appear 
below  as  percentage 
distributions  of  university 
research  projects  among  all  ROAR 
projects  within  each  of  the 
twelve  broad  acquisition 
categories.  The  categories  are 
arrayed  from  the  category  for 
which  the  incidence  of 
university  projects  was  highest 
to  the  category  of  lowest 
incidence.  No  prior  assumptions 
were  made  about  the  data 
distribution  within  categories. 
The  average  incidence  of  370 
university  projects  among  all 
876  ROAR  projects  was  42%. 


Commercial  Products 
&  Practices 

Tot  Proj  Univ  Proj  Incidence 
113  61  54% 
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Tot  Proj  Univ  Proj  Incidence 
90  25  28% 


Supports  Systems 

Tot  Proj  Univ  Proj  Incidence 
269  71  26% 


Streamlining 
Acquisition  Momt 


Tot  Proj  Univ  Proj  Incidence 
140  32  23% 


A  general  observation  about 
these  data  is  that  university 
research  projects  seem  to  be 
relatively  skewed  toward  the 
acquisition  policy  issues  that 
are  concerned  with  manufacturing 
changes  to  improve  U.S. 
competitiveness;  with  the  need 
to  adjust  to  the  political 
imperatives  of  reduced  defense 
spending;  and  with  the 
realignment  of  the  defense 
industrial  base. 

The  skew  in  research  priorities 
might  prove  to  be  a  fortunate 
development  for  universities  and 
the  nation:  the  current 
administration  apparently  aims 
to  speed  defense  spending 
reductions  while  trying  to 
smooth  conversion  of  defense 
industry  to  competitive 
commercial  pursuits. 


CONCLUSIONS  AND  IMPLICATIONS  FOR 
DEFENSE  ACQUISITION  POLICY 
RESEARCH 


The  conclusions  of  this  paper 
are  that  (1)  universities 
currently  conduct  a  large  number 
of  defense  acquisition  policy 
studies;  (2)  they  have  not 
depended  directly  on  defense 
funds  for  most  of  their  work; 

(3)  the  DoD-funded  acquisition 
research  community  is  not  well 
aware  of  university  efforts  in 
progress,  and  vice  versa;  and 

(4)  many  university  researchers 
may  have  anticipated  the  types 
of  acquisition  research  efforts 
needed  to  support  key  U.S. 
policy  trends  in  the  1990s. 

In  view  of  proposed  DoD  budget 
reductions  in  coming  years,  it 
seems  likely  that  DoD-funded 
research  organizations  will  be 
competing  vigorously  with  all 
other  DoD  activities  for 
declining  resources. 

But  ROAR'S  survey  of  ongoing 
studies  suggests  that  vast 
resources  for  defense 
acquisition  policy  research 
exist  in  universities,  where 
they  are  not  subject  to  the  same 
budget  forces  as  the  DoD  faces. 
Moreover,  there  is  tentative 
evidence  that  the  DoD,  Commerce 
Department,  and  National  Science 
Foundation  plan  to  augment  these 
resources  with  new  funding  for 
additional  acquisition-related 
projects  to  be  performed  by 
universities  and  other  agents 
outside  the  Defense 
Department . [ 7 ] 


For  within-DoD  research 
organizations,  these  additional 
findings  and  observations 
suggest  important  opportunities 
as  well  as  potential  new 
problems . 


These  portents  commend  a  shift 
in  operating  style  for  DoD- 
funded  acquisition  research 
organizations,  toward  a  strategy 
that  might  be  called  wide-area 
research  management. 
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wide-area  research  management 
recognizes  that  some  DoD 
organizations  will  not  continue 
to  employ  as  many  researchers  as 
they  do  now.  If  resources  for 
research  remain  abundant  outside 
the  DoD,  then  defense 
institutes,  schools,  and  centers 
may  organize  themselves  so  that 
they  become  authorities  on  the 
current  state  of  acquisition 
studies  everywhere  —  especially 
in  universities. 

By  attaining  unmatched 
expertise,  these  organizations 
will  be  positioned  to  guide 
policymakers  in  allocating 
scarce  DoD  research  funds 
effectively,  often  including  the 
direction  of  department  funding 
to  researchers  outside  the 
traditional  defense  fold  to 
satisfy  DoD  netv  ». 

Soon,  for  example,  fewer  defense 
organizations  may  be  able  to 
say,  "We  just  received  a  vital 
new  acquisition  priority  plus  a 
million  in  DoD  money;  let's  get 
a  few  analysts  downstairs 
started  on  it  for  a  year." 
Instead,  a  successful 
organization  may  be  one  that  can 
quickly  find  those  campuses 
where  professors  are  already 
working  on  a  related  problem, 
can  persuade  the  scholars  to 
focus  additionally  on  DoD's 
specific  priority  for  $50,000, 
and  then  will  work  with  them  to 
get  the  results  out  in  months. 

If  defense  research 
organizations  decide  to 
specialize  in  wide-area  research 
management,,  they  will  need  to 
know  much  more  about  the 
research  activities  of  the 
nation's  universities  than  they 
do  today. 

There  will  be  perils  in  fitting 
non-DoD  resources  to  DoD 


research  priorities.  But  there 
are  also  perils  in  operating  in 
ways  that  fail  to  fit  defense 
resources. 


FOOTNOTES 

1.  Defense  acquisition  policies 
delineate  between  acceptable  and 
unacceptable  management 
practices  for  controlling  how 
the  DoD  acquires  goods  and 
services.  Defense  acquisition 
policy  research  is  defined  here 
as  research  that  improves 
understanding  about  effective 
practices  at  various  levels  of 
control  and  also  about  internal 
and  external  circumstances  that 
can  cause  the  DoD  to  change  its 
policies.  Some  policies 
describe  practices  that  are 
unique  to  DoD.  Others  deal  with 
practices  that  are  commonly  used 
in  less-tightly  regulated 
organizations  such  as  companies 
and  non-DoD  government  agencies. 

2.  Part  of  the  annual 
publications  output  is  currently 
reported  in  Defense  Logistics 
Studies  Information  Exchange, 
1993  Annual  Department  of 


Defense 

Biblioaraohv 

of 

Loaistics 

Studies  and  Related 

Documents.  Deoartment 

Defense,  1993,  pp.  1-663. 

of 

The 

volume  lists  over  1400  titles 
produced  by  or  for  DoD  each 
year.  Although  less  than  half 
of  the  logistics-related  reports 
listed  are  considered  by  the 
author  to  involve  acquisition 
policy  research,  logistics 
concerns  form  only  one  of  many 
areas  germane  to  acquisition 
policy. 

3.  Ibid.,  pp.  664-661.  For 
example,  23  logistics-related 
projects  were  performed  by 
universities  out  of  2,805  total 
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projects  completed  and  published 
with  DoD  funding  during  a  two- 
year  period.  Also,  the  IDA 
(Institute  for  Defense  Analyses) 
Cost  Research  Symposium  began  an 
innovative  program  in  the  early 
1980s  to  identify  and 
disseminate  news  about  ongoing 
research  on  cost-estimating 
models  and  other  cost  issues  in 
DoD  acquisition.  In  1989  IDA 
reported  209  projects  to  the 
Symposium's  within-DoD 
participants.  No  university 
research  was  included;  all 
reported  projects  in  1989  were 
performed  by  within-DoD 
organizations.  See  S.J.  Balut 
and  K.L.  Wilson,  "The  IDA  Cost 
Research  Symposium,"  Institute 
■Fnr  Defence  Analyses,  IDA 
Document  D-647,  August  1989, 
various  pages. 

4.  Levels  of  DoD  funding  for 
university  projects  were  not 
discovered  during  the 
interviews.  Some  forms  of 
support  may  have  included 
sharing  of  DoD  empirical  data 
and  privileged  access  to  records 
and\or  facilities  rather  than 
direct  funding  for  the  projects. 

5.  The  PA&E  initiative  has 
begun  yielding  acquisition 
policy  research  publications  by 
university  scholars.  Some 
studies  are  authored  by 
professors  and  published  by 
RAND.  For  example,  see 
Rogerson,  W.P.,  "An  Economic 
Framework  for  Analyzing  DoD 
Profit  Policy,"  The  RAND 
Corporation,  R-3860,  1992;  and 
Rogerson,  W.P.,  "Overhead 
Allocation  and  Incentives  for 
Cost  Minimization  in  Defense 
Procurement,"  The  RAND 
Corporation,  R-4013,  1992. 
Rogerson  is  professor  of 
economics  at  Northwestern 
University. 


6.  It  is  possible  that  some 
research  projects  benefit  from 
university  facilities  and  staff 
that  are  funded  by  non¬ 
acquisition  DoD  organizations. 
For  example,  the  DoD  funds 
university  research  in  basic  and 
applied  sciences  through  offices 
such  as  the  Advanced  Research 
Projects  Agency,  the  Office  of 
Naval  Research,  and  the  Air 
Force  Office  of  Scientific 
Research.  But  it  is  not 
apparent  that  these  DoD 
resources  contribute  even 
indirectly  to  the  financing  of 
university  acquisition  research. 

7.  For  example,  DoD's  Advanced 
Research  Projects  Agency  (ARPA) 
will  manage  $472  million  in  1993 
for  a cqu i s i t i on -r e 1 a t ed 
research,  technology 
development,  training,  and 
education  through  its  new 
Technology  Information  Project. 
Universities  are  eligible  to 
receive  up  to  $356  million  of 
the  total  figure  (and  one-half 
of  the  $356  million  is  reserved 
exclusively  for  universities) . 
DoD's  internal  research 
facilities  are  eligible  to 
receive  no  more  than  $91  million 
of  the  total,  and  they  must 
compete  for  these  funds  against 
other  government  agencies, 
universities,  for-profit  firms, 
and  non-profit  organizations 
that  are  also  eligible.  See 
Technology  Information  Project, 
"Program  Information  Package  for 
Defense  Technology  Conversion, 
Reinvestment,  and  Transition 
Assistance,"  Advanced  Research 
Projects  Agency,  Department  of 
Defense,  March  10,  1993,  various 
pages.  ARPA  is  a  DoD  research 
organization  with  decades  of 
experience  in  successful  "wide- 
area  management"  of  university 
research  in  basic  and  applied 
sciences  to  meet  defense 
priorities. 
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APPENDIX  1 


CURRENT  STATUS  OF  THE  ROAR  ELECTRONIC  INFORMATION  SYSTEM 


Rollout  of  DSMC^s  Research  on  Ongoing  Research  fROAR^  System 

In  the  fall  of  1992  the  Defense  Systems  Management  College  (DSMC) 
completed  the  development  and  testing  of  ROAR'S  electronic 
information  system.  On  October  19,  1992,  the  system  was  opened  to 
acquisition  researchers  working  at  42  institutions  where  400 
ongoing  projects  in  defense  acquisition  policy  research  had 
previously  been  identified  by  ROAR  interviews. 

By  mid-1993,  ROAR  will  be  open  to  all  researchers,  including  those 
in  over  200  institutions,  both  within-DoD  and  at  universities, 
where  more  than  1,000  acquisition  studies  are  known  to  be  underway. 


User  Access  to  ROAR 

Anyone  can  access  the  ROAR  system  by  calling  a  commercial  telephone 
number  using  a  computer  and  modem,  plus  any  standard  communications 
software  program  (e.g.  Procomm,  CrossTalk,  etc.).  ROAR'S  phone 
number  and  communications  settings  are  as  follows: 


Phone  No.:  (703)  805-3981 


Baud  rate 
Data  bits 
Parity: 
Stop  bits 


2400-1200 
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If  the  caller  has  not  previously  registered  an  abstract  of  ongoing 
acquisition  policy  research,  ROAR  explains  how  to  "download”  basic 
instructions  that  provide  an  overview  of  ROAR'S  services  plus 
directions  on  how  to  register  an  abstract  of  ongoing  research. 


When  the  user  has  read  the  instructions  and  is  ready  to  register  an 
abstract  of  ongoing  research,  he  or  she  calls  ROAR  again.  Then  the 
user  creates  a  unique  ID,  types  in  basic  descriptive  information 
(name,  organization,  phone  number,  etc.)  and  enters  an  abstract  of 
the  research  project  in  normal  English  words  and  sentences. 
Finally,  the  user  is  instructed  to  log-off  (terminate  the  phone 
connection)  and  call  back  48  hours  later. 


By  the  time  the  user  calls  back,  an  electronic  "mailbox"  will  exist 
for  the  user's  unique  ID.  After  logging-on,  instructions  appear 
that  explain  how  the  user  can  download  the  contents  of  the  mailbox 
to  his  or  her  computer.  The  mailbox  contains  a  copy  of  the  user's 
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own  abstract,  search  results  (the  ten  abstracts  in  ROAR'S  database 
that  most  closely  match  the  project  that  the  user  registered) ,  and 
further  directions. 

The  directions  (which  the  user  prints  out  after  downloading  the 
mailbox  contents  from  ROAR  and  logging-off)  explain  that  ROAR  will 
conduct  additional  searches  against  the  user's  abstract  after  ROAR 
has  received  by  postal  mail  the  user's  signed  statement  certifying 
that  the  registration  data,  with  changes  marked  clearly,  are 
accurate  in  all  respects. 

As  soon  as  ROAR  receives  the  certified  registration,  the  database 
is  searched  again.  The  results  (the  next  ten  abstracts  that  most 
closely  match  the  user's  project  description)  are  loaded  in  the 
user's  mailbox. 


ROAR'S  Weekly  Service  for  Researchers 

After  the  user  has  downloaded  these  additional  search  results,  ROAR 
begins  its  weekly  service  to  the  user.  Every  Monday  morning,  the 
user's  mailbox  is  loaded  with  one  new  abstract  that  has  not  been 
presented  before.  The  abstract  is  generated  by  a  weekend  search  of 
the  entire  database  for  the  next  most-closely-matching  project 
description. 

Since  ROAR  acquires  abstracts  continuously,  the  weekly  service 
means  ROAR  can  keep  informing  its  users-researchers  individually 
about  the  latest  relevant  ongoing  work  throughout  the  active  life 
of  their  research  projects.  It  does  this  automatically  without 
requiring  each  user-researcher  to  repeat  the  registration  process. 

The  key  to  the  weekly  service  is  ROAR'S  searching  methods. 
Searching  is  based  on  the  user's  registered  abstract,  and  the  work 
is  done  off-line  by  ROAR'S  computer  programs.  The  user  can  call 
ROAR  once  a  week  (at  his  or  her  convenience)  to  download  the  search 
results.  Each  user  spends  less  than  one  minute  (the  time  spent 
logging  on,  downloading,  and  ending  the  call)  to  stay  informed 
about  all  relevant  new  research  underway. 

ROAR'S  active  users  report  that  these  services  are  enabling  them  to 
discover  relevant  ongoing  studies  about  which  they  might  otherwise 
know  nothing.  Users  decide  for  themselves  whether  they  want  to  act 
on  their  discoveries  by  calling  one  another  to  exchange  ideas  and 
data. 

The  ROAR  electronic  system  is  open  to  the  public  every  weekday  from 
6:00  a.m.  to  2:00  p.m.  and  3:00  p.m  to  11:00  p.m. ,  eastern  time. 

Individuals  can  also  register  new  projects  by  calling  ROAR'S 
contract  staff  in  person  (voice  only)  at  (703)  271-5988,  Mon-Fri, 
and  answering  questions  about  their  project.  However,  ROAR 
database  search  results  are  available  only  by  downloading  with  use 
of  a  computer  and  modem.  The  FJV  Company  is  ROAR's  contractor. 
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THE  UNCERTAIN  VALIDITY  OF  RESEARCH  RESULTS 

by 

Henry  C.  Alberts 

Defense  Systems  Management  College,  Fort  Belvoir,  Virginia 


ABSTRACT 

One  major  difference  between  the  physical  and  social 
sciences  is  the  way  in  which  investigative  boundaries 
can  be  selected.  In  physics  and  mathematics,  one  can 
isolate  particular  events  and  study  them  with  the 
assurance  that  results  observed  are  likely  to  be 
repeatable.  In  the  social  sciences,  boundary  selection 
might  be  dictated  by  factors  other  than  experimental 
convenience,  or  even  the  possibility  of  achieving 
experimental  validity;  and  the  results  obtained  may  be 
neither  repeatable  nor  reproducible  across  time  and 
space.  It  is  specifically  these  issues  of  boundary 
setting  and  result  variability  that  create  both  the 
richness  for  inquiry  and  the  challenges  to 
understanding  of  exactly  what  has  been  learned.  This 
paper  presents  some  hypotheses  about  the  conditions 
under  which  social  systems,  and  complex  engineering 
systems  might  be  better  investigated. 

THE  BEGINNING  BOUNDARIES 

A  fundamental  principle  of  what  has  become  known 
as  "science"  is  the  ability  to  reproduce  results  across 
time  and  space.  One  of  the  factors  which  influenced 
my  choice  of  physics  and  mathematics  as  subjects  of 
study  was  that  assurance  of  result  reproducibility  - 
one  could  come  to  know  the  "right"  answer  to 
properly  posed  questions. 

As  first  an  experimental,  and  then  theoretical 
physicist  and  mathematician,  I  was  delighted  to  find 
that  properly  constructed  expenments  would  transfer 
to  other  locations.  The  work  I  was  doing  in 
supersonic  aerodynamics  in  1949  could  be  checked 
using  results  produced  by  others  with  similar 
equipment  in  other  places.  In  those  days,  meetings  of 
the  American  Physical  Society  were  places  where 
those  of  us  working  on  the  same  problems  could  get 
together  and  compare  experimental  results.  Because 
results  were  the  same  (to  within  what  we  knew  to  be 
experimental  error)  we  reassured  ourselves  about  the 
"correctness"  of  our  knowledge;  and  validated  our 
work. 


From  time  to  time,  someone  would  report  on  a  result 
no  one  had  obtained  before.  In  almost  every  case,  the 
experimenter  had  changed  the  regime  (the 
boundaries)  in  which  prior  work  had  been  done.  For 
example:  in  the  early  19S0’s  a  few  of  us  were 
exploring  the  physics  of  supersonic  flow.  Wind 
tunnels,  which  were  the  standard  test  environment  for 
aerodynamic  testing,  had  severe  limitations  for  such 
work.  Attempts  to  create  supersonic  wind  speeds  had 
resulted  in  damage  to  the  fans  and  propeller 
mechanisms  which  produced  the  air  flow;  and  to  the 
tunnel  walls.  The  requirement  to  pass  through  a 
turbulent  trans-sonic  regime  damaged  the  propellers, 
and  the  heat  generated  down  stream  from  them 
stressed  both  the  tunnel  wall  material  and  the 
instrumentation  we  could  build  at  that  time.  The 
invention  of  the  "Shock  Tube’  at  Johns  Hopkins 
University  Applied  Physics  Laboratory  provided  a 
means  to  study  supersonic  flow  without  generating 
those  problems. 

The  idea  of  the  shock  tube  was  to  create  a  "period  of 
supersonic  flow",  a  transient  rather  than  an  extended 
time  event.  Because  the  creation  of  the  burst  of  flow 
was  instantaneous  and  only  lasted  a  few  milliseconds, 
the  problems  of  experimental  equipment  construction 
and  instrumentation  became  tractable.  A  shock  tube 
consisted  of  two  chambers  isolated  from  one  another 
by  a  piece  of  material  called  a  diaphragm.  One 
chamber  was  maintained  at  normal  room 
temperatures  and  pressures  and  air  was  free  to  flow 
through  it.  The  second  chamber  (the  enclosed  space) 
contained  gas  under  high  compression.  The 
diaphragm  maintained  the  separate  integrity  of  both 
compartments.  When  the  diaphragm  was  pierced,  the 
compressed  gas  would  press  outward  from  its 
restricted  chamber,  bend  back  the  diaphragm,  and 
pass  into  the  second  chamber.  Since  gases  are 
compressible  fluids,  it  took  some  time  for  conditions 
in  both  portions  of  the  tube  to  stabilize  at  room 
conditions.  As  the  gas  under  high  pressure  entered 
the  tube,  it  formed  a  "shock  front"  (or  wave)  behind 
which  supersonic  flow  occurred.  Figure  1  presents  a 
typical  shock  wave  and  indicates  the  regions  of 
supersonic  and  transonic  flows. 
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TYPICAL  SHOCK  WAVE  PRESSURE  OVER  TIME 


0  50  100  150  200  250  300  350 

TIME  IN  MILLISECONDS 

—  PRESSURE  LEVEL 


FIGURE  1 


Of  course,  heat  was  still  produced,  but  it  was  of  such 
short  duration  that  none  of  the  system  components, 
the  shock  tube  material,  the  instrumentation,  or  the 
models  we  used  were  damaged. 

Moreover,  although  our  supersonic  regime  lasted  for 
only  very  .short  periods,  what  we  learned  was  truly 
representative  of  supersonic  flow.  The  rea.son  was 
because  the  time  required  to  respond  to  the  impulse 
behind  the  shock  front  and  establish  stable  supersonic 
flow  around  the  model  was  quite  short  compared  to 
the  time  the  .supersonic  regime  la.sted.  That  is  to  say; 
the  response  limes  of  the  solid  material  and  the  fluids 
to  the  shock  wave  and  its  associated  flow  were 
different  enough  to  permit  studies  of  the  supersonic 
flow  phenomena.  Those  studies  yielded  crucial 
information  about  the  conditions  to  which  bodies 
returning  from  space  orbits  would  be  subjected. 


At  one  meeting  of  the  Physical  Society.  Arthur 
Kantrowitz  reported  results  from  shock  tube 
experiments  which  differed  significantly  from  what 
we  had  come  to  believe  was  "the  norm".  His 
contribution  was  to  change  the  boundaries  of  the 
experimentalSystem.  He  pre-heated  his  compre.s.sed 
gas  beyond  the  normal  temperatures  caused  by  gas 
compression;  and  he  changed  the  gas  as  well.  As  a 
result,  the  temperatures  behind  the  shock  front  were 
highly  elevated.  Under  these  conditions,  although  the 
models  remained  the  same,  there  were  differences  in 
the  flow  patterns  because  the  gas  became  ionized. 
Many  of  us  were  fascinated  by  Kantrowitz’s  results 
especially  because  ionization  and  molecular 
dissociation  could  very  easily  explain  small 
differences  in  flow  patterns  we  had  already  ob,ser\  ed 
and  categorized  as  being  part  of  the  "system 
experimental  error". 
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But  the  most  interesting  thing  about  all  of  this  was 
that  when  real  re-entry  bodies  were  tested,  those  test 
results  verified  the  laboratory  results:  The  flow 
patterns  around  re-entering  ''chicles  were  like  those 
obtained  in  shock  tube  experiments  until  the 
temperature  built  up  in  the  mid-level  re-entry.  At  mid 
level,  the  atmosphere  ionized  around  the  vehicle 
leading  to  what  we  came  to  call  "communications 
blackout":  a  situation  marked  by  inability  to  send 
electromagnetic  signals  through  the  atmosphere 
because  it  simply  reflected  from  the  ionized  layer 
which  surrounded  the  incoming  vehicle.  This 
phenomena  corresponded  to  the  well-known 


communica'ions  phenomena  which  had  become 
known  as  "ionospheric  scatter":  radio  waves  would 
reflect  off  the  ionized  layers  of  the  earth’s 
atmosphere.  Ionospheric  scatter  conununication 
systems  used  this  effect  to  achieve  long  range  high- 
frequency  radio  communications. 

There  are  many  more  instances  in  physics  and 
engineering  within  my  own  45  years  of  experience 
which  demonstrate  consistently  that  changes  in 
experimental  system  boundaries  cause  results  which 
differ  from  previously  accepted  norms.'  The  point  of 
this  discussion  is  to  lay  a  basis  for  the  assertion  that: 


When  measuring  the  behavior  of  what  we  call  "stable"  entities  under  changing 
environmental  conditions,  not  only  should  changes  to  behavior  be  expected,  but 
one  should  also  anticipate  change  to  the  entities  under  study.  That  is;  data 
obtained  by  repeating  the  same  set  of  experiments  within  even  slightly  changed 
"experimental  systems"  can  yield  very  different  and  sometimes  unpredictable 
results! 


THE  MIDDLE  LEVEL  BOUNDARIES 

Physicists  try  to  establish  tight  control  over 
experimental  systems  and  their  boundaries.  That  can 
be  done  in  many  cases  because  physical  environments 
and  equipment  do  noi  "learn"  or  "change"  of  their 
own  volition  while  undergoing  physical 
experimentation.  Social  Science  experimentation  is  a 
considerably  different  thing. 

I  began  to  work  with  large,  integrated  systems  in 
1953.  Initially  my  work  with  Ballistic  Missile  and 
Space  programs  was  in  the  aerodynamic  design  area. 
But  incipient  success  quickly  led  us  to  problems  in 
the  system's  operational  use  and  logistics  planning.  It 
became  necessary  to  know  how  inaccuracies  in  our 
knowledge  of  where  targets  were  located  interacted 
with  uncertainties  about  distances  between  continents 
and  errors  caused  by  imperfect  system  performance 


to  produce  errors  in  mis.sile  impact  locations. 
Continuing  questions  about  system  performance  led 
to  experimental  testing  of  weapon  effectiveness  when 
different  weapon  mixes  were  employed  by  differing 
groups  of  human  beings  acting  either  as  individuals 
or  within  coordinated  organizations.^ 

By  the  late  1950’s  and  early  1960’s,  there  was 
considerable  interest  in  developing  combat  tactics  for 
small  groups  of  soldiers  operating  in  a  guerilla 
environment.  It  was  that  particular  work  which 
provided  the  first  stmetured  insights  into  problems  of 
experimental  data  reproducibility  when  human  beings 
were  involved. 

Considerable  amounts  of  this  sort  of  work  were  done 
at  the  Army's  Combat  Development  Experimentation 
Center  (CDEC)  at  Fort  Hunter-Liggett  and  Fort  Ord 
in  Central  California.  The  Army  established  CDEC 
in  1958  as  a  place  to  develop  new  tactics  and 
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doctrines  which  made  best  use  of  newly  developing 
weapon  systems.  One  of  the  ways  we  did  experiments 
to  arrive  at  suggested  force  structural  change  was  to 
devise  and  repeat  exercises  which  would  produce 
statistically  rigorous  results.  We  would  repeat 
portions  of  these  exercises  using  different  groups  of 
people:  a  process  we  referred  to  as  "replication".* 
When  we  had  developed  enough  data,  our  analysis 
would  provide  performance  "norms"  which  were  then 
used  to  design  tactics. 

We  had  some  problems  with  that  approach. 
Sometimes  the  results  of  a  carefully  designed  "set- 
piece"  of  action  performed  by  one  group  differed  so 
much  from  the  results  obtained  from  other  groups 
that  we  had  to  exclude  one  or  more  result  sets  from 
the  complete  set  of  trial  data.  Our  usual  justification 
was  that  "something  had  gone  amiss  during  that 
particular  run";  and  the  data  from  that  run  was 
"rogue  data",  "outside  the  norm".^ 

We  also  discovered  that  when  we  asked  groups  of 
soldiers  to  do  a  set  of  activities,  in  different 
sequences,  the  results  would  very  often  differ  by 
more  than  we  anticipated.  That  is,  if  we  designed  an 
experiment  consisting  of  five  tasks  which  could  be 
performed  in  any  order,  and  we  set  about  collecting 
performance  data  for  each  of  them,  we  found  that 
task  performance  results  tended  to  vary  depending  on 
task  performance  sequence.  To  state  this  point 
another  way; 


If  a  Task  Sequence  of  1,  2,  3,  4,  5  yielded  results: 
1|>  2|,  3i,  4|,  5i; 

and  Task  Sequence  of  2,  5,  4,  1,3  yielded  results: 
I’,  22,  32,  42,  52; 

and  Task  Sequence  of  I,  S,  2,  4,  3  yielded  results: 
l3»  23,  33,  43,  53 

we  found  that  for  each  task, 

(li>  l2>  •••  ^3,  53) 

performance  varied  so  widely  that  sinqile  statistical 
treatment  was  inappropriate.  Wie  had  observed  non¬ 
reproducibility! 

Clearly,  how  people  performed  tasks  differed 
depending  upon  the  sequence  of  task  performance: 
they  "learned"  something  by  performing  a  task  which 
affected  how  they  performed  succeeding  tasks.* 
Thus,  in  order  to  define  performance  in  a  way  that 
would  be  reproducible  across  time  and  space,  a 
matrix  array  of  tasks  and  trials  would  need  sufficient 
entries  in  each  cell  to  insure  that  the  effects  of  task 
sequence  on  result  obtained  was  known  and  could  be 
accounted  for. 

A  five  task  matrix  array  shown  below  indicates  some 
variable  task  order  choices.  The  objective  of  the  task 
sequencing  is  to  have  tasks  performed  in  differing 
orders  and  to  permit  two  instances  when  task  orders 
followed  each  other  in  sequence.* 


FIVE  TASKS  TAKEN  IN  VARIED  SEQUENCE  WITHIN  5  TRIALS 


TASK/I^ 

TRIAL  1 

TRIAL  2 

TRIAL  3 

TRIAL  4 

TRIAL  5 

1 

1st 

2nd 

3rd 

4th 

5th 

2 

2nd 

5th 

1st 

3rd 

4th 

3 

3rd 

Lst 

4th 

5th 

3rd 

4 

4th 

3rd 

5th 

2nd 

2nd 

5 

5th 

4th 

2nd 

1st 

1st 
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To  get  high  assurance  of  statistical  validity  of  results 
obtained  would  require  performance  of  a  great  many 
trials.  In  fact,  many  complex  task  sequences  would 
require  more  players  and  tests  than  could  be 
practically  realized. 

In  1965  I  became  interested  in  whether  this  problem 
of  result  non-repeatability  was  unique  to  our 
particular  application^  or  applied  in  general  to 
complex  systems.  To  find  out,  I  did  some  research 
for  the  U.S.  Commissioner  of  Education*  on  the 
"Conditions  of  Learning".  The  problem  was  to 
construct  an  experimental  program  to  determine  how 
conditions  within  the  classroom  affected  learning.  We 
would  measure  learning  by  administering  an  objective 
knowledge  or  skill  test. 

A  literature  review  quickly  provided  more  than  a 
thousand  references  which  purported  to  have 
performed  a  study  of  how  conditions  under  which 
learning  occurred  affected  the  learning  outcome.  As 
each  of  the  reports  was  studied,  it  became  clear  than 
none  of  them  had  recognized  the  richness  of  the 
problem! 

To  treat  all  of  the  variables  which  might  affect  the 
learning  process  in  a  classroom,  one  would  have  to 
consider  at  least  the  conditions  listed  below: 

1.  The  purpose  for  which  that  class  was 
convened:  General  Education,  Training  for 
Specific  Task  Performance... 

2.  General  Educational  Institutional  sponsor. 

Commercial  Work  Sponsor,  Military 

Sponsor. . . 

3.  The  kind  of  facility  in  which  the  class  was 

held:  Typical  Public/Private  School  Room, 
Industrial  Training  Classroom,  Military 

Classroom,  Special  Purpose  Facility  (which 
had  multiple  uses  other  than  classes)... 

4.  The  Size,  Shape,  Configuration,  of  Facilities 
used... 

5.  The  methodology  of  instruction:  Lecture, 

Demonstration,  Discovery,  Computer... 


6.  The  kind  of  instructor:  Subject  Professional 
Instructor,  General  Professional  Instructor, 
Expert  General  Practitioner,  Practitioner 
with  special  experience... 

7.  The  subject  matter  of  instruction: 

fundamental  categories  of  subject  matter 
(mathematics,  economics,  psychology); 
some  other  specialized  subject  matter 
pointed  at  specific  sets  of  task  performance 
levels;  or  subject  matter  whose  objective 

was  to  build  conditioned  responses 

(behavioral  modification)... 

8.  The  class  demography  pattern:  ages  and 

numbers  of  students,  past  learning  and  life 
performance,  socioeconomic  grouping, 

language  and  literature  skills... 

These  eight  major  categories  which  we  thought  "had 
impact  on  the  problem"  turned  quickly  into  a  matrix 
containing  over  10,000  cells  when  all  of  the  detailed 
indentures  under  each  of  the  eight  categories  were 
considered!  When  we  tried  to  structure  a  program  to 
derive  statistically  significant  data,  we  came  to 

believe  there  weren’t  sufficient  people  available  to 
create  a  full  data  set.’ 

Even  more  important,  we  believed  that  any 

measurements  we  did  take  would  change  with  time. 
The  performance  of  each  student  would  be  affected 
by  all  the  prior  experiences  of  that  student  as 
modified  by  the  instructional  experience  in  the 
classroom.  That  meant  that  in  such  a  complex 
situation,  we  couldn’t  see  how  to  build  a  continually 
growing  data  base.  Therefore  we  couldn’t  take  a  few 
students  in  each  category,  complete  the  work,  and 
then  repeat  the  work  sequence  with  some  other 
students  in  other  categories.  During  the  time  wc  were 
performing  the  first  experiment,  the  students  who  had 
not  yet  participated  in  it  would  be  experiencing  things 
in  their  lives  which  affected  their  performance  when 
they  came  to  the  classroom  later. 

We  had  inadvertently  constructed  a  real  experiment 
for  which  available  time  and  resources  might  always 
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be  excessive!  There  should  be  no  misunderstanding, 
the  Commissioner  of  Education  although  stunned  by 
the  implications  of  our  analysis  agreed  with  us  that  as 
constructed,  the  problem  was  "too  large  to  solve"!  He 
had  no  suggestions  for  smaller  studies  because  both 
he  and  we  understood  that  the  variabilities  in  results 
over  time  would  very  likely  dominate  the  process  and 
thus  make  any  rigorous  analysis  extremely  (perhaps 
too)  difficult. 


GREATLY  EXPANDED,  CROSS- 
CULTURAL  LATER  STAGE 
BOUNDARIES 


In  1967,  I  was  asked  to  serve  as  a  management 
consultant  to  Swedish  industrialists.  The  leading 
commercial  bank  in  Scandinavia'"  was  "involved" 
with  almost  every  large  Swedish  industrial 
organization."  My  task  was  to  address  losses 
(negative  profits)  being  experienced  by  a  number  of 
the  companies  and  to  "turn  those  companies  around" 
so  they  made  a  reasonable  profit. 

As  I  looked  at  the  way  business  was  done  in  the 
largest  paper  making  facility  in  Europe,  I  became 
convinced  that  much  of  the  difficulty  was  caused  by 
"cultural"  or  "mind-set"  patterns  within  the  Swedish 
and  European  cultures.  Acceptable  ways  of  doing 
business  were  not  amenable  to  change.  In  fact,  one  of 
Chief  Executive  Officer  (CEO)  refused  to  accept 
recommendations  for  making  his  company  profitable 
because  they  were  "counter  to  established  Swedish 
cultural  practice".  Of  course,  since  bankers  were  in 
charge,  he  was  released  from  the  company  and 
replaced  by  someone  who  did  accept  those  changes. 
The  changes  suggested  turned  the  company’s  Fiscal 
Year  1967  loss  of  35  million  Kroner  to  a  Fiscal  Year 
1968  profit  of  30  million  Kroner. 

The  lesson  learned  was  that  cross  cultural  influences 
have  deep  affect  on  both  how  people  perform  similar 
(perhaps  even  identical)  tasks  and  the  depth  of  their 
resistance  to  necessary  change.  In  a  seminal  study. 


Gervaise  Busche'^  found  that  even  after  extensive 
Total  Quality  Management  training,  and  acceptance 
of  that  concept,  the  management  in  a  number  of 
American  auto  manufacturing  plants  did  not  want  the 
cultural  changes  associated  with  worker 
empowerment.  But  even  more  important,  they  did  not 
"understand  why  we  would  pay  someone  such  a  large 
salary  to  prevent  problems  when  we  haven’t  had  any 
problems  in  a  couple  of  years." 

In  1978,  I  was  asked  to  visit  Sweden  again  to  meet 
with  the  Board  of  Directors  of  Rank-Xerox  in 
Stockholm.  A  friend  was  on  the  board  and  said  he 
had  been  impressed  by  the  work  done  in  1966-1968, 
and  thought  it  would  be  instructive  to  hear  my  later 
perceptions  of  how  Sweden  was  progressing. 

I  found  much  which  surprised  me.  The  culture  had 
changed!  What  would  have  been  unacceptable  in  the 
mid  to  late  1960’s  was,  in  the  mid  1970’s,  not  only 
acceptable  but  preferred.  One  of  the  most  interesting 
changes  was  how  income  maintenance  programs  were 
conceived.  From  the  time  of  their  institution  in  the 
1930’s,  income  maintenance  was  provided  without 
regard  to  societal  contributions  recipients  made. 
Many  simply  remained  in  their  state  provided  homes, 
making  no  effort  to  participate  in  working  society. 
That  had  still  been  the  case  when  1  lived  in  Sweden 
from  1967-1968.  But  early  in  the  1970’s  the 
government  had  begun  a  new  approach  to  income 
maintenance.  Those  receiving  it  were  encouraged  to 
open  businesses  and  address  themselves  to  creating 
some  good  or  service.  If  the  income  derived  from 
those  businesses  was  less  than  the  amount  which 
would  have  been  received  as  income  maintenance,  it 
would  be  augmented  to  insure  at  least  the  stipulated 
amount  was  received.  The  effect  of  this  policy  shift 
was  to  create  new  small  businesses  which  rented 
space  and  involved  themselves  in  commerce.  The 
small  entrepreneur  was  protected  from  gross 
economic  failure,  but  the  effect  of  involving 
previously  untapped  individual  creative  capacities 
greatly  benefitted  both  to  the  society  and  the 
individuals;  creativity  flourished  and  many  kinds  of 
hand  made  goods  were  available  in  many  revitalized 
central  city  market  districts. 
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When  I  asked  what  caused  the  change,  I  was  told  that 
it  came  about  because  people  recognized  the  need  to 
encourage  independence  rather  than  dependence. 

I  thought  Sweden  was  unique  until  I  went  to  the 
People's  Republic  of  China  in  1983  and  found  the 
same  kind  of  entrepreneurial  encouragement  program 
there.  China  had  experienced  a  remarkable  decrease 
in  the  numbers  of  factory  jobs  required  to  meet 
production  quotas.  Rather  than  make  income 
maintenance  payments  with  no  return  to  the  State,  the 
government  gave  unemployed  individuals  sufficient 
money  to  set  up  small  service  businesses.  Those 
business  were  "unregulated"'’  and  entrepreneurs 
could  keep  much  of  what  they  made.'^  By  1985 
many  of  those  businesses  had  flourished  and  some 
entrepreneurs  had  become  "millionaires"  in  China. 
Unfortunately  later  events  contracted,  and  sometimes 
even  eliminated,  the  entrepreneurial  sectors  which  we 
had  seen  in  Shanghai,  Nanjing  and  Beijing  in  1983. 


SOME  IDEAS  ABOUT  VALIDITY 
DURATIONS 


I  believe  that  as  problems  become  more  complex, 
their  boundaries  widen.  That  is  especially  true  when 
the  problems  concern  large  systems  which  include 
human  beings  as  crucial  portions  of  interconnected 
physical  systems.  We  have  developed  a  number  of 
techniques  which  can  define  envelopes  of  operating 
integrity  experimentally  for  rather  complex 
devices.”  Those  envelopes  can  be  reproduced  over 
time  and  across  space.  But  we  have  yet  to  establish 
comparable  capabilities  for  projecting  how  human 
learning  or  cultural  changes  that  happen  during  the 
course  of  a  complex  investigation  will  affect 
investigatory  results.  Rather,  the  opposite  is  true:  We 
have  considerable  difficulty  projecting  how 
individuals  will  make  use  of  complex  "systems'  in 
complicated  and  changing  situations.  It  simply  isn’t 
easy  to  predict  accurately  what  humans  will  do  when 
faced  with  changing  circumstances.” 


It  seems  to  me  that  one  important  indication  of  the 
durability  of  research  result  validity  can  come  from 
understanding  the  concept  of  "time  constants  of 
change". 

In  studies  we  performed  for  the  Chief  of  Naval 
Operations,  10  models  of  the  U.S.  economy  were 
examined  to  see  how  well  their  predictions  matched 
actual  economic  change  over  time  periods  of  from  3 
months  to  up  to  10  years.  Actual  data  for  40  quarters 
were  taken  from  Bureau  of  Labor  Statistics  and 
Federal  Reserve  Board  data  bases  and  input  to  each 
model.  The  resulting  quarterly  forecasts  were  then 
matched  against  the  economy’s  actual  performance 
for  each  succeeding  quarter.  We  found  the  percentage 
error  between  each  model’s  forecast  and  the  actual 
result.  The  forecast  errors  for  each  model  were 
plotted  against  quarterly  forecast  periods.  Figure  2 
plots  the  envelope  of  forecast  errors  for  the  entire  set 
of  models. 

Figure  2  shows  us  the  percentage  error  between  the 
forecast  and  actual  performance  for  quarterly  periods 
which  had  elapsed  since  the  forecast  was  made. 

It  appears  from  Figure  2  that  all  models  provide 
forecasts  which  are  accurate  to  within  ±  .5%  of  the 
actual  results  for  periods  within  18  months  of  the  date 
of  forecast.  After  that,  forecast  accuracy  rapidly 
deteriorated  so  that  at  24  months,  forecast  errors  are 
up  to  5%;  and  by  36  months,  errors  are  more  than 
20%.  All  models  display  these  forecast  limitations. 
What  the  data  indicate  is  that  the  time  constant  of 
change  for  the  U.S.  economy  may  be  of  the  order  of 
18  months.  That  is,  one  (or  more)  event(s)  occur 
which  affects  some  part  of  the  economy,  and  18 
months  later  effects  are  seen  on  the  gross  economic 
performance.” 

There  is  some  reinforcement  of  that  insight.  In 
looking  at  change  to  the  Consumer  Price  Index  (CPI) 
during  the  1970’s,  we  found  that  as  the  price  of 
energy  changed  remarkably,  the  structural  balances'* 
of  CPI  component  elements  did  not  reach  stability  for 
some  time. 
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%  DIFFERENCE  BETWEEN  FORECAST  AND 
ACTUAL  ECONOMIC  PERFORMANCE 


%  ERROR  OF  FORECASTS 


QUARTERS  BETWEEN  FORECAST  AND  COMPARISON 

—  BOUNDARY  OF  +  ERRORS  BOUNDARY  OF  -  ERRORS 

FIGURE  2 


In  fact,  increases  in  oil  prices  migrated  through  the 
CPI  in  about  18  months.” 

An  additional  observation  might  be  derived  by 
looking  at  economic  events  which  occurred  after 
enactment  of  the  Tax  Reform  Act  of  1986.  The  result 
of  a  major  change  in  Limited  Real  Estate  Partnership 
tax  policy  was  lowered  economic  value  of  those 
partnerships.^  Banks  which  held  loan  portfolios 
which  were  heavily  weighted  with  limited  partnership 
real  estate  ventures  experienced  devalued  security 


against  outstanding  loans.  As  accumulation  of  value 
reductions  continued  for  an  extended  period,  some 
banking  institutions  found  that  not  only  did  their 
portfolios  no  longer  support  the  amount  of  loans 
outstanding  against  them,  but  also  that  the  differmces 
between  debt  and  collateral  value  approached  the 
institution’s  net  capitalization.  Whoi  that  occurred, 
the  institution  was  no  longer  considered  viable.  If 
additional  capitalization  was  not  quickly  forthcoming, 
the  institution  was  liable  to  seizure  by  Federal  or 
State  banking  examiners.  In  regions  where  real  estate 
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had  been  the  "engine"  driving  "economic  growth", 
the  leveraged  effect  on  other  business  created 
significant  economic  downturns.  A  chief  example  was 
the  New  England  Region  centered  around  Boston, 
Massachusetts,  extending  south  to  Hartford. 
Connecticut,  and  North  to  Portland,  Maine.  Another 
is  the  State  of  California  which  is  currently 
experiencing  a  recession  based  on  both  real  estate 
investment  value  depreciation  and  defense  spending 
reductions.  The  banking  problems  became  visible  in 
1988  as  did  the  economic  recession. 


THE  PERSPECTIVE  OF  EXPANDING 
BOUNDARIES  OF  INVESTIGATION 


To  develop  a  perspective  of  how  changing  boundaries 
of  investigation  and  time  constants  of  change  affect 
the  validity  of  research  results,  consider  the  following 
argument: 

■  The  Shock  Tube  Illustration:  The  time 
required  for  creating  major  temperature  stress  in  a 
shock  tube  wall  was  very  much  longer  than  the  high 
temperature  impulse  generated  by  creation  and 
passage  of  the  shock  front  and  its  associated  flow. 
But  the  time  necessary  to  establish  a  stable  supersonic 
flow  regime  around  the  model  was  much  less  than  the 
duration  of  that  flow  generated  behind  the  shock 
wave.  Since  both  time  constants  were  in  "the  proper 
relationship",  reproducible  data  was  collected  over 
long  periods  of  time.  This  is  a  case  of  "tightly 
focused"  experimentation  with  a  narrow  boundary. 

■  The  Army  Tactics  and  Doctrine 
Illustration:  The  time  required  to  make  the 
appropriate  numbers  of  measurements,  analyze  and 
interpret  the  experimental  results  of  tactics  and 
doctrine  development  was  quite  long.  When  using  the 
same  players,  as  the  structure  and  order  of  the  task 
sequences  used  changed,  performance  from  trial  to 
trial  changed  too:  that  is,  results  became  increasingly 
less  reproducible  as  the  experimentation  process 
proceeded.  Later  tests  using  other  groups  of  players 
did  not  generally  reproduce  previous  results.  It 


seemed  that  the  time  constants  of  player  learning 
were  much  shorter  than  the  time  required  to  complete 
the  experimentation  for  this  quite  large  problem 
boundary.  The  hypothesis  was:  when  a  boundary 
became  large  enough  to  encompass  interrelated 
people  and  equipment,  it  also  became  "too  wide"  to 
permit  gathering  reproducible  data  within  the  time 
period  required. 

■  The  Educational  Conditions  Illustration: 
The  estimated  time  required  to  collect  sufficient  data 
to  address  the  question  of  how  conditions  of  learning 
affect  the  actual  learning  achieved  was  of  the  order  of 
18  months.  So  many  variables  were  incorporated 
within  the  boundaries  required  to  generate  an 
understanding  of  the  question  that  even  if  sufficient 
population  could  be  developed,  the  time  required  to 
make  the  measurements  and  the  effects  of  making 
them^'  very  likely  would  permit  sufficient  cultural 
change  and  learning  outside  the  bounds  of  the 
experimental  situation  to  mitigate  against  result 
stability  and  reproducibility.  Because  the  time 
constants  of  human  learning  appeared  to  be  shorter 
than  the  time  required  to  understand  them,  we 
believed  that  in  this  wide  boundary  case,  any  results 
we  achieved  would  likely  be  of  short  temporal 
validity. 

■  The  U.  S.  Economy  Illustration:  The 
boundary  of  the  U.S.  economy  is  as  large  as  the 
country  itself.  Furthermore,  the  effects  we  seek  to 
understand  are  generated  by  accumulations  of  the 
individual  actions  of  large  numbers  of  people. 
Because  the  event  of  change  is  perceived  differently 
by  individuals,  and  because  all  individuals’  responses 
must  aggregate  to  make  their  effect  felt,  the  time 
constants  of  stimulative  change  are  much  shorter  than 
those  associated  with  aggregated  responses.  Although 
that  relationship  introduces  stability  in  the  forecasting 
process,  it  limits  very  much  the  time  period  over 
which  that  stability  can  be  expected  to  persist. 

In  other  words,  the  question  being  asked  is: 

Do  differences  in 

*  complexities  of  structure  inherent  in  broadly 
based  systems  considered  as  a  single  entity, 
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*  human  capacity  to  ieam  while  working 
within  broadly  bounded  systems,  and 

•  the  'time  constants  of  change”  for  each 
"system  component", 

make  achievement  of  "reproducible"  observations  of 
broadly  bounded,  complex  systems,  over  time  and 
space  so  difficult  as  to  be  unlikely? 

THE  IMPLICATIONS  OF  THIS  PAPER 


When  we  seek  "scientific"  research  results  which  are 
reproducible  across  time  and  space,  we  must  look 
very  carefully  at  the  data  we  obtain  from 
investigations  of  broadly  bounded,  complex  problems. 

Research  into  complex  systems  behavior  may  last 
over  long  time  periods,  perhaps  years.  During  that 
period,  changes  may  have  occurred  to  one  or  more 
fundamental  elements  within  the  complex  system,  or 
individuals  within  the  system  may  have  adjusted  their 
behavior.  The  environments  within  which  the 
research  was  conducted  may  even  have  become 
unstable  or  even  have  disappeared.  Completed 
research  results  may  quickly  be  overtaken  by  events 
if  the  research  program  has  continued  for  longer  time 
periods  than  the  research  environment  or  participants 
responses  can  be  expected  to  remain  stable. 

It  may  be  that  some  research  results  for  broadly 
bounded,  complex  systems  have  only  transient 
validity. “  If  that  were  the  case,  it  would  imply  that 
achieving  data  reproducibility  ("scientific"  validity) 
would  limit  research  programs  to  relatively  short 
durations  within  rather  narrow  boundaries.  It  might 
also  imply  that  knowledge  obtained  from  such 
research  activities  should  be  used  before  its  validity 
is  reduced' 

But  it  may  also  be  easier  to  devise  research  designs 
for  problems  with  narrower  foci  of  study;  studies 
which  can  be  completed  within  short  enough  time 
periods  to  preclude  much  change  in  what  we  measure 


or  in  the  environment  which  embeds  the  problem. 
But  when  we  study  complex  interacting  man-machine 
systems  which  must  necessarily  involve  large 
experimental  populations,  there  are  so  many 
possibilities  for  change  that  achieving  unchanging 
research  results  may  be  beyond  the  capabilities  of  our 
current  knowledge  and  methodology  base. 

If  one  thinks  about  decomposing  broadly  bounded 
problems  to  achieve  narrow  focus  components,  the 
problem  becomes  one  of  correctly  structuring 
interactive  knowledge  from  decomposed  elements. 
Though  that  methodology  might  achieve  time 
constants  of  change  which  help  extend  result  validity 
for  the  decomposed  units,  the  advantage  might  not 
extend  to  the  imputed  total  interactive  structure.  The 
data  we  do  obtain  may  not  retain  its  validity  over  a 
long  enough  time  to  be  applied  usefully. 

In  fact,  any  narrowing  of  the  boundaries  and  scope  of 
research  inquiry  to  achieve  "scientific  validity"  may 
exclude  many  avenues  of  investigation  crucial  to  a 
nation’s  cultural  development. 

In  short,  taken  as  a  set,  the  hypotheses  presented  here 
might  require  us  to 

•  recognize  the  need  for  research  designs 
which  take  account  of  how  the  scope  of  the 
investigation  affects  the  time  periods  during 
which  results  will  be  valid;  and,  perhaps 
even  more  importantly, 

•  reexamine  our  belief  that  once  a  research 
program  has  been  accomplished,  we  can 
institutionalize  actions  based  on  those 
results. 

The  latter  point  is  of  great  potential  importance  to 
those  who  make  policy.  If  research  results  were  to  be 
of  uncertain  and  transient  validity,  then  their  use  as 
the  basis  for  detailed  institutionalization  may 
encourage  (or  even  insure)  responses  which  might  be 
most  appropriate  to  solve  problems  of  the  past  within 
the  environment  of  that  time.  The  solutions  might  be 
much  less  suitable  to  solve  current  problems  within  a 
quite  different  current  environment. 
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1.  The  case  can  be  made  that  Newtonian  physics  treated  large  aggregations  of 
particles  (or  molecules)  and  the  rules  which  were  derived  to  explain  behavior  of  large 
integrated  bodies  broke  down  when  the  body's  constituent,  smaller  particles  were 
considered.  The  need  to  deal  with  particulate  "body”  behaviors  led  first  to 
development  of  duality  concepts  for  electromagnetic  radiation  in  the  visible  spectrum: 
(that  is,  light  behaves  as  if  it  were  both  a  wave  and  particles),  thence  to  quantum 
theory  fox  groups  of  particulate  matter  acting  together  within  similar  environment,  and 
finally  to  quantum  electrodynamics  which  deals  with  characteristics  of  individual 
particles  acting  individually  within  an  environment. 

2.  Results  of  that  experimentation  determined  defense  force  configurations  for  many 
years. 

3.  We  might  decide  that  testing  a  new  tank's  effectiveness  required  a  platoon  of  new 
tanks  to  oppose  a  platoon  of  old  ones  in  a  set  of  one  or  more  battle  scenarios.  We 
would  repeat  them  using  different  soldiers.  The  number  of  times  we  repeated  each 
experimental  sequence  and  the  numbers  of  different  soldiers  used  were  derived  from 
statistics  of  stochastic  processes.  The  usual  result  was  a  set  of  experimental  matrices 
with  numbers  of  replications  stipulated  for  each  matrix  cell. 

4.  "Everyone  knows"  that  rogue  data  is  not  useful  for  statistical  analysis. 

5. 1  came  to  recognize  later  that  when  dealing  with  systems  within  which  humans  are 
included,  the  process  which  governs  system  performance  seems  to  obey  a  rule  which 
can  be  stated  as:  Task  performances  i  observe  today  integrate  the  total  life  experience 
up  until  today. 

6.  The  two  repeated  sequences.  Task  3  followed  by  Task  4  (in  trials  1  and  2)  and 
Task  5  followed  by  Task  2  (in  trials  3  and  4)  offer  a  chance  to  test  the  hypothesis 
that  results  when  tasks  are  performed  in  particular  sequences  will  be  closer  than  will 
results  when  those  same  tasks  are  performed  in  a  variable  set  of  sequences. 

7.  That  is,  the  Army's  Experimentation  process  as  practiced  by  CDEC  in  formulating 
tactics  and  doctrine. 

8.  At  that  time,  Nicholas  Katzenbach. 
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9.  That  is,  the  numbers  of  participants  necessary  to  provide  data  of  sufficient 
statistical  robustness  was  very  large  indeed;  and  it  would  be  very  difficult  to  find 
relationships  resulting  from  so  complex  a  matrix  array. 

10.  The  Stockholms  Enskilda  Bank  was  then  owned  by  the  seven  Swedish  industrial 
families  which  included  the  Soderbergs,  the  Wallenbergs,  and  Axel  Johnson.  It  has 
since  been  merged  with  the  Sveriges  Handlesbankto  form  one  of  the  largest  industrial 
and  credit  banks  in  Europe. 

1 1 .  These  included  ASEA  (now  ASEA-Brown-Bovary),  Electrolux  AB  (the  world-wide 
vacuum  cleaner  company),  AVESTA  Industrie  (specialty  steel  mills  throughout 
Europe),  Scandinavian  Airlines  System,  and  Axel  Johnson  Industries  (Shipping  and 
transportation,  light  manufacturing  throughout  Europe)  and  others. 

12.  Associate  Professor,  Simon  Fraser  University,  Vancouver  British  Columbia, 
Canada. 

1 3.  That  is,  the  owners  rather  than  the  State  would  set  sale  prices  for  items  produced 
and  sold. 

14.  The  businesses  and  the  entrepreneurs  were  taxed  at  the  normal  rates  however 
and  the  State  recovered  funds  from  those  taxes. 

1 5.  Performance  envelopes  for  tanks,  aircraft,  and  ships  are  commonplace  and  are  the 
bases  for  complex  training  devices. 

16.  Psychologists  have  achieved  considerable  success  in  analyzing  observed 
behavioral  patterns  of  selected  groups  of  individuals  to  determine  "normal  behavior" 
for  those  groups.  There  has  been  only  limited  success  in  predicting  what  individuals 
will  do  under  hypothesized  circumstances.  There  is  even  some  difficulty  in  predicting 
how  long  a  group  behavior,  once  observed,  will  be  sustained;  especially  under 
conditions  of  rapid  change  within  the  society. 

17.  See  the  report,  "Accuracy  of  Economic  Forecasts",  General  Research  Corporation 
Report  to  the  Director  of  Compensation,  U.S.  Navy,  Office  of  the  Chief  of  Naval 
Operations,  1979.  Since  economics  depends  heavily  on  how  large  number  of 
individuals  respond  over  time  we  came  to  believe  that  it  took  some  time  before 
individual  "people"  response  changes  accumulated  into  measurable  effects  in  a  macro- 
economic  model. 
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18.  The  Consumer  Price  Index  is  composed  of  a  number  of  sectors  such  as  "energy", 
"housing",  "food",  "medical  expenses",  and  so  forth.  When  the  economy  is  "stable", 
the  relative  values  of  each  of  these  segments  with  respect  to  all  other  segments 
remains  reasonably  constant  to  within  small  percentage  change  boundaries.  Usually, 
major  change  in  one  sector  results  in  adjustment  of  other  sectors  over  time. 

19.  See  "Rate  of  Change  of  the  Consumer  Price  Index  in  Response  to  Exogenous 
Economic  Events";  General  Research  Corporation  Report,  1980 

20.  In  many  cases,  the  value  of  the  real  estate  was  essentially  the  value  of  the  tax 
shelter  the  partnership  created. 

21 .  Heisenberg's  uncertainty  principle,  loosely  interpreted,  can  be  seen  as  stating  that 
the  making  of  measurements  disturbs  the  situation  being  measured;  thus  there  are 
limits  to  the  things  we  can  know  about  complex  situations. 

22.  That  is,  they  are  truly  valid  only  during  the  research  process  itself. 
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DISCLAIMER  NOTICE 

Views  expressed  in  this  paper 
are  the  authors'  and  are  not 
necessarily  shared  by  the 
United  States  Air  Force  or  the 
Department  of  Defense. 


ABSTRACT 

The  purpose  of  this  paper 
is  to  present  a  system  of 
methods,  principles,  and  rules 
that  will  aid  in  the  satis¬ 
factory  accomplishment  of  the 
Cost  and  Operational  Effec¬ 
tiveness  Analysis  (COEA) .  This 
paper  provides  a  brief  intro¬ 
duction  to  the  COEA  process, 
describes  when  a  COEA  is 
required,  how  COEAs  are  used, 
how  to  prepare  for  the  COEA, 
describes  the  analysis  process 
while  focusing  on  the  basic 
elements  of  the  COEA, 
discusses  organizational  res¬ 
ponsibilities,  the  report 
format,  and  concludes  with  a 
summary  of  the  COEA  review 
process.  The  general  framework 
for  this  paper  is  based  on  the 
Office  of  Assistant  Secretary 
of  Defense  COEA  Guide,  Cost 
and  Operational  Effectiveness 
Analysis  (COEA)  and  Air  Force 
Materiel  Command  (AFMC) 


Pamphlet  173-1,  AFMC  Cost  & 
Operational  Effectiveness 
Analysis  (COEA)  Guide. 

INTRODUCTION 

For  most  acquisition  pro¬ 
grams,  COEAs  provide  critical 
information  essential  to  the 
decision  making  process.  The 
COEA  evaluates  the  costs  and 
operational  effectiveness  of 
alternative  courses  of  action 
to  meet  recognized  defense 
needs.  The  COEA  is  a  required 
DOD  systems  acquisition 
analysis  that  provides  cost 
and  operational  effectiveness 
data  for  a  particular  concept 
under  consideration.  COEA  data 
developed  during  the  analysis 
of  the  concept  or  system 
enhancement  are  compared  to 
similar  data  from  alternative 
systems  or  system  enhancements 
under  consideration.  This 
process  assures  that  only  the 
most  cost  and  operationally 
effective  systems  are  brought 
into  the  military  inventory. 

The  COEA  will  typically 
draw  on  several  sub-analyses. 
These  include  analyses  of 
mission  needs,  the  threat (s) 
and  U.S.  capabilities,  the 
interrelationship  of  systems, 
the  contribution  of  multi-role 
systems,  measures  of  effec¬ 
tiveness,  measures  of  per¬ 
formance,  logistics,  costs, 
and  cost-effectiveness 
comparisons.  Collectively, 
these  sub-analyses  provide  the 
information  necessary  to 
answer  the  many  questions 
generated  by  the  acquisition 
management  system  process. 

The  DOD  acquisition 
management  system  is  comprised 
of  five  phases  linked  by 
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decision  points  or  milestones. 
At  each  milestone,  the 
decision  to  proceed  is  based 
on  information  developed 
during  the  previous  phase.  As 
the  process  proceeds  from 
concept  definition,  through 
demonstration  and  validation, 
development,  production  and 
deployment,  to  operations  and 
support  (O&S) ,  the  milestone 
reviews  aim  to  ensure  that  the 
program  will  satisfy  user 
needs  and  remain  within  the 
approved  program  effectiveness 
and  financial  constraints. 
Figure  1  depicts  the  current 
acquisition  milestones  and 
phases . 

DOD  Instruction  5000.2, 
dated  23  February  91,  mandates 
COEAs  for  ACAT  I  programs  at 
Defense  Acquisition  Board 
(DAB)  Milestones  I  and  II 
reviews.  COEAs  may  also  be 
required  for  Milestones  III 
and  IV.  The  COEA  provides 
vital  information  to  decision 
makers  and  supporting  docu¬ 
mentation  for  these  critical 
milestone  decisions. 


THE  COEA  REQUIREMENT 

All  acquisition  programs 
are  based  on  mission  needs 
identified  by  the  Unified  and 
Specified  Commands,  Military 
Departments,  OSD,  developers 
and  other  relevant  organ¬ 
izations.  Needs  can  be  based 
on  deficiencies,  oppor¬ 
tunities,  or  obsolescence 
issues.  The  needs  are 
identified  by  assessing  cur¬ 
rent  and  projected  threats  and 
taking  into  account  the 
changing  national  defense 
policy.  The  COEA,  an  analysis 
based  on  mission  needs, 
considers  all  of  these 
factors . 

COEAs  are  required  for 
DOD  Acquisition  Category 
(ACAT)  I  programs  and  may  be 
required  for  ACAT  II,  III,  and 
IV  programs.  They  provide  com¬ 
parative  analyses  of  the  costs 
and  operational  effectiveness 
of  alternative  solutions  in¬ 
tended  to  satisfy  an  estab¬ 
lished  mission  need.  The 
information  and  supporting 
documentation  that  results 
from  the  COEA  process  is 
critical  for  selecting  the 
best  possible  system  to 
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satisfy  user  requirements. 
Each  COEA  will  deal  with  a 
different  system,  and  depend¬ 
ing  on  the  program  size  and 
the  phase  of  the  development 
process,  may  differ  signif¬ 
icantly  in  scope. 

Responsibility  for  the  pre¬ 
paration  of  COEAs  usually 
rests  with  the  Military 
Department  or  Defense  Agency 
responsible  for  the  mission 
area  encompassing  the  iden¬ 
tified  mission  need.  A  lead 
service  is  designated  for 
joint  programs;  the  Joint 
Staff  may  be  designated  as 
lead  in  certain  cases.  For  Air 
Force  programs,  the  Office  of 
Assistant  Secretary  of  the  Air 
Force  for  Acquisition  (ASAF/A) 
gave  the  responsibility  for 
preparing  Phase  0  and  Phase  I 
COEAs  to  the  operational  com¬ 
mands,  with  support  and 
assistance  of  the  implementing 
command  (Action  Memo,  10  Nov 
89,  ASAF/A) . 

HOW  COEAs  ARE  USED 

COEAs  are  submitted  by  the 
military  services  at  dej ig- 
nated  milestones  in  the  acqui¬ 
sition  cycle.  They  provide  an 
evaluation  of  the  costs  and 
benefits — the  operational  ef¬ 
fectiveness  or  military  util¬ 
ity — of  alternative  courses  of 
action  to  meet  recognized 
defense  needs.  One  of  the 
alternatives  typically  rep¬ 
resents  the  current  program  or 
status  quo.  Another  is  usually 
an  improved  version  of  the 
current  program.  Other  alter¬ 
natives  are  assessed  against 
these  cases  in  terms  of  their 
changes  in  cost  and  effec¬ 
tiveness.  COEAs  provide  infor¬ 
mation  on  the  sensitivity  of 
alternatives  to  potential 


changes  in  key  assumptions, 
variables,  and  constraints. 
They  aid  decision  making, 
facilitate  communications 
among  OSD,  service  principals 
and  staffs,  and  document 
acquisition  decisions  by 
providing  a  record  of  the 
alternatives  considered.  COEAs 
should  be  homogenous  with 
other  acquisition  support  doc¬ 
umentation  such  as  the  Mission 
Need  Statement  (MNS)  and  the 
Operational  Requirements  Doc¬ 
ument  (ORD) . 

COEAs  are  one  of  many 
required  supporting  documents 
in  the  complex  acquisition 
process.  They  are  required 
prior  to  major  decision  points 
by  the  Defense  Acquisition 
Board  (DAB)  at  Milestones  I, 
II,  and  IV.  The  DAD  may  also 
require  COEAs  at  Milestone  III 
as  program  changes  warrant.  In 
addition  to  providing  insight 
into  the  relative  advan¬ 
tages/disadvantages  of  the 
many  alternatives  being  com¬ 
pared,  the  COEA  facilitates 
discussions  between  the  major 
participants  (OSD,  AF,  and  the 
using  and  implementing  com¬ 
mands)  at  early  stages  to 
settle  concerns  and/or  dis¬ 
agreements.  The  COEA  documents 
the  analytical  process  and 
provides  data  for  milestone 
inputs  and  decisions. 

COEA  PREPARATION 

This  section  describes 
the  activities  that  occur  in 
the  COEA  process  from  Pre- 
Milestone  0  through  Phase  III. 
It  also  provides  discussion  of 
the  study  plan  developed  at 
Phase  0.  The  greatest  emphasis 
in  this  section  is  on  Phase  0, 
because  the  activities  that 
occur  in  this  phase  are 
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replicated  to  a  greater  or 
lesser  degree  in  the  sub¬ 
sequent  phases. 

PRE-MILESTONE  0  -  The  needs 
identified  in  the  Mission  Area 
Assessment  (MAA)  and  the 
supporting  Mission  Need 
Analysis  (MNA)  may  come  from  a 
variety  of  sources  within  or 
without  the  operating  command. 
Deficiencies  that  can  be 
satisfied  by  changes  in 
doctrine,  tactics,  training, 
or  organization  are  sent  to 
the  military  department  for 
consideration  and  action. 
Deficiencies  that  could  poten¬ 
tially  result  in  the  estab¬ 
lishment  of  a  new  or  enhanced 
program  are  documented  in  a 
Mission  Need  Statement  (MNS) , 
and  the  process  proceeds  to  a 
Milestone  0  decision.  The 
individuals  and  organizations 
responsible  for  preparing  the 
COEA  should  be  aware  of  the 
issues  raised  during  this 
phase  of  the  acquisition 
process . 

PHASE  0  -  Phase  0  COEA 
activities  are  illustrated  in 
Figure  2.  The  study  plan  that 
is  developed  and  the  analyt¬ 


ical  process  that  occurs 
during  this  phase  culminate  in 
the  report  used  for  the  Mile¬ 
stone  I  decision. 

PHASE  I  -  During  Phase  I, 
the  demonstration  and 
validation  phase,  the  COEA 
plan  must  be  updated  to 
identify  the  Phase  I  analysis. 
The  COEA  prepared  during  this 
phase  can  be  much  more 
detailed  than  the  previous 
phase.  There  are  fewer 
alternatives,  which  can  be 
described  with  more  confidence 
in  terms  of  cost  and  per¬ 
formance.  The  demonstration 
and  validation  tests  performed 
on  the  alternatives  will 
provide  useful  data  for  the 
analysis . 

The  COEA  analysis  should 
once  more  visit  the  go/no  go 
question  (see  Figure  2) , 
making  use  of  the  better 
alternative  definitions  and 
any  new  information.  Assuming 
that  the  acquisition  process 
is  to  proceed,  the  goal  of 
this  COEA  is  to  identify  the 
preferred  alternative.  Sensi¬ 
tivity  analyses  should  quan¬ 
tify  the  impact  on  cost 
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Figure  2.  Phase  0  COEA  Activities 
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effectiveness  of  uncertainties 
in  cost,  performance,  support- 
ability,  and  schedule.  The 
analysis  should  identify  cost 
ceilings  and  performance 
floors . 

Figure  3  highlights  the 
differences  between  Phases  0 
and  I  in  the  application  of 
COEA  elements.  The  system 
alternatives  in  Phase  I 
typically  represent  a  narrower 
range,  with  more  information 
and  definition.  Therefore,  the 
analysis  can  be  more  detailed 
and  specific.  The  additional 
information  should  permit  much 
better  cost  and  performance 
estimates  of  the  alternatives. 
Point  estimates  should  be  made 
and  bounded  by  a  range  of 
uncertainty.  To  lessen  uncer¬ 
tainty  and  to  accomplish 
relative  life  cycle  cost 
comparisons,  the  data  should 
incorporate  any  available 
engineering  or  "bottom-up" 
costing  method.  However,  since 
life  cycle  costing  is  seldom  a 
pure  engineering  estimate,  a 
combination  of  engineering, 
analogy,  and  parametric  cost 
estimates  is  usually  neces¬ 
sary  . 
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Figure  3.  Comparison  of  the 
Application  of  the  Key 
Elements  in  Phase  0  and  I 
COEAs 
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PHASES  II  AND  III  -  Figure 
4  illustrates  the  activities 
in  Phases  II  and  III.  In 
preparation  for  Milestone  III, 
the  acquisition  decision 
authority  may  require  that  the 
COEA  be  updated  to  account  for 
any  factors  which  may  have 
changed  during  the  preceding 
phase.  If  a  major  modification 
is  anticipated,  the  decision 
authority  may  require  that  a 
new  COEA  be  performed  in 
preparation  for  Milestone  IV. 
The  requirements  for  this  COEA 
will  be  delineated  in  the 
planning  phase. 


STUDY  PLAN  -  The  first  task 
of  COEA  preparation  is  the 
development  of  a  study  plan. 
While  this  plan  focuses  on  the 
Phase  0  COEA,  it  forms  the 
basis  for  the  Phase  I  and 
subsequent  COEAs  as  well.  The 
plan  should  include  discussion 
of  the  mission  need,  threat, 
environment,  constraints  and 
assumptions,  scenarios,  con¬ 
cept  of  operations,  des¬ 
cription  of  alternatives, 
methodology,  functional  ob¬ 
jectives,  measures  of  effec¬ 
tiveness,  measures  of  per¬ 
formance,  logistics,  acquis¬ 
ition  cost  (R&D  and 
investment) ,  operations  and 
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support  cost,  disposal  cost, 
military  construction  cost, 
and  models  and  data.  Figure  5 
presents  a  suggested  study 
plan. 
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Figure  5.  Suggested  COEA  Study 
Plan  Format 

The  study  plan  should 
outline  an  approach  (method¬ 
ology)  for  performing  the 
analysis,  identify  tasks  to  be 
performed,  and  present  a 
schedule  for  completing  the 
effort.  The  detail  required  in 
the  study  plan  may  vary 
depending  on  the  size  of  the 
anticipated  development  pro¬ 
gram.  The  plan  should  be 
prepared  by  the  operating 
command  leading  the  effort, 
with  support  from  the 
implementing  command  (usually 
AFMC)  and  the  support  agencies 
(AFOTEC,  AFSAA,  AFCAA,  AFMC  as 
appropriate) .  The  plan  should 
be  coordinated  with  the 
implementing  command's  DCS  for 
Requirements,  the  appropriate 
support  agencies  and  the 
SAF/AQ  and,  for  ACAT  ID 
programs,  be  reviewed  with  the 
OASD/PA&E.  This  will  help 
ensure  that  all  logical. 


viable  alternatives  get  a  fair 
assessment  and  that  mission 
area  parochialism  is  kept  at  a 
minimum.  The  plan  should  be 
maintained  and  updated 
throughout  the  COEA  process. 

ANALYSIS  PROCESS 

The  purpose  of  the  analysis 
is  to  identify  the  strengths 
and  weaknesses  of  the  alter¬ 
natives,  determine  costs,  and 
to  ultimately  identify  the 
preferred  solution.  The 
analysis  process,  as  outlined 
in  the  study  plan,  may  be 
performed  by  in-house  study 
groups,  by  or  with  the  support 
of  the  AFMC  product  and 
logistics  centers,  or  by 
contractor  teams  if  funding  is 
available.  Several  of  the  COEA 
elements  listed  in  the  study 
plan  can  be  categorized  as 
administrative:  the  study 
team,  the  study  advisory 
group,  the  report  format,  the 
schedule,  and  the  review 
process.  These  who  and  when 
elements  will  be  discussed  in 
the  sections  addressing  organ¬ 
izational  responsibilities, 
report  format,  schedule,  and 
the  review  process.  The  bulk 
of  the  elements  listed  in  the 
study  plan  are  analytical 
elements  in  that  they  help 
determine  what  will  occur  in 
the  COEA  process.  These 
elements  will  be  discussed  in 
turn  under  the  categories  of 
acquisition  issues,  alter¬ 
natives,  and  analysis.  Addi¬ 
tional  information  on  the  COEA 
elements  is  contained  in 
Attachment  2  of  AFMC  Pamphlet 
173-1.  The  following  para¬ 
graphs  address  critical 
elements  of  the  analysis 
process . 
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ACQUISITION  ISSUES 

MISSION  NEED  -  It  normally 
will  not  be  necessary  to  apply 
extensive  effort  to  develop  or 
rejustify  the  mission  need.  It 
may  be  sufficient  to  review 
the  original  justification. 
Some  questions  to  consider: 

■  Is  the  need  derived 
from  and  compatible  with 
higher  echelon  operations? 

■  Was  it  derived  from  a 
deficiency  that  is  likely  to 
occur  over  a  broad  spectrum  of 
conditions  or  will  the  need 
arise  only  in  unique  or 
unlikely  circumstances? 

■  Was  it  based  on 
comparisons  of  a  valid  threat 
and  friendly  capabilities 
projected  within  the  same  time 
frame? 

■  Have  any  changes 
occurred  either  in  the  threat, 
world  situation,  strategy,  or 
technology  capabilities  that 
could  have  significant  impact 
on  the  need? 

THREAT  -  Describe  the 
threat  in  sufficient  detail  so 
that  the  effect  of  the 
alternatives  can  be  assessed. 
Always  address  the  following: 

■  Get  the  intelligence 
community  (operating  command, 
DIA,  USAF/IN,  etc.)  involved 
early  to  confirm  the  threat. 

■  Include  a  range  of 
threats  to  account  for 
uncertainties  in  the  esti¬ 
mates. 

■  Include  any  limitations 
that  will  impact  threat  effec¬ 
tiveness,  such  as  logistics, 
limited  operational  capa¬ 
bilities,  strategy  or  tactics, 
and  technology. 

■  Aim  for  a  realistic 
threat  description;  avoid 


overestimating  the  threat 
capabilities . 

ENVIRONMENT  -  The  nature  of 
the  need  will  dictate  the 
susceptibility  of  the  analysis 
to  environmental  influences. 
For  example,  a  surveillance 
system  may  be  impacted  greatly 
by  the  weather,  while  a 
communications  system  may 
experience  relatively  little 
impact.  When  considering 
environmental  factors,  one 
should  always: 

■  Identify  those  factors 
that  will  significantly  impact 
the  developmental  system. 

■  Identify  the  range  of 
the  factors  that  could  be 
expected. 

CONSTRAINTS  AND  ASSUMPTIONS 
-  Identify  the  constraints  and 
assumptions  that  will  signifi¬ 
cantly  impact  the  COEA.  They 
form  one  basis  for  the  sensi¬ 
tivity  analyses  that  must  be 
performed.  Key  constraints  and 
assumptions  to  be  considered 
are: 

■  Limitations  on  person¬ 
nel  available  for  either  oper¬ 
ation  of  the  system  or  its 
support. 

■  Limitations  on  funds 
available  for  development, 
investment,  or  O&S. 

■  Any  obvious  techno¬ 
logical  limitations. 

■  Limitations  imposed  by 
schedule  requirements  or  lo¬ 
gistic  constraints. 

■  Limitations  imposed  by 
environmental,  treaty,  legal, 
or  political  constraints. 

SCENARIOS  -  Thoroughly 
define  and  develop  the 
scenarios  in  which  the 
alternatives  will  operate.  The 
scenarios  draw  on  the  need, 
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the  threat  estimates,  the 
operational  concepts,  and  the 
environmental  constraints.  One 
of  the  scenarios  should  be 
identified  as  the  baseline. 
Consider  the  following: 

■  The  scenarios  should 
describe  both  friendly  and 
enemy  forces  in  terms  of  their 
numbers,  capabilities,  deploy¬ 
ments,  and  employment  con¬ 
cepts  . 

■  Include  environmental 
factors  that  impact  opera¬ 
tions  . 

■  Include  logistic 
factors  that  impact  opera¬ 
tions  . 

■  The  baseline  scenario 
should  comply  with  the 
Illustrative  Planning  Sce¬ 
nario  . 

m  Other  scenarios  can  be 
developed  to  investigate  the 
robustness  of  the  alternatives 
to  changes  in  constraints  and 
assumptions,  threat,  etc. 

ALTERNATIVES  -  Alternatives 
must  address  the  concept  of 
operations  (CONORS)  as  well  as 
describe  the  alternatives. 

CONCEPT  OF  OPERATIONS 
Develop  a  concept  of  opera¬ 
tions  for  each  alternative.  In 
doing  this  one  should: 

■  Describe  how  the  system 
will  be  deployed  during 
operations . 

■  Identify  staffing 
levels  to  operate  and  support 
the  system. 

■  Describe  how  the  system 
will  be  operated  and  main¬ 
tained  and  its  interactions 
with  other  operating  systems. 

■  Describe  how  the  system 
fits  into  the  existing  command 
and  control  system.  Identify 
any  unique  requirements. 

32 


■  Describe  the  basing 
mode  and  any  unique  require¬ 
ments  for  basing,  logistics, 
support,  etc. 

■  Identify  organic  and 
contractor  support  require¬ 
ments  . 

DESCRIPTION  OF  ALTERNATIVES 

The  set  of  alternatives 
should  contain  the  present 
capability  within  the  current 
organizational  structure  as 
the  base  case  or  reference 
alternative  and  incorporate 
the  following  as  appropriate: 

■  Include  the  current 
system  as  base  case. 

■  Use  the  current  system 
with  product  improvement (s) . 

■  Include  systems  cur¬ 
rently  under  development. 

■  Add  conceptual  systems, 
if  available  in  the  required 
time  frame. 

■  Include  systems  of 
other  services  or  foreign 
governments. 

■  Use  commercial  systems, 
if  applicable. 

Include  a  manageable  range 
of  alternatives.  Avoid  in¬ 
cluding  alternatives  that  are 
obviously  unsuitable.  If  there 
is  a  question  as  to  the 
suitability  of  an  alternative, 
include  it.  On  the  other  hand, 
if  rough  order  of  magnitude 
(ROM)  estimates  of  cost  or 
performance  indicate  an  alter¬ 
native  is  clearly  unsuitable, 
the  estimates  need  not  be 
refined.  The  subsequent 
analysis  should  resolve  the 
issue.  Allow  for  new  or 
modified  alternatives  to  be 
added  as  the  analysis  proceeds 
and  new  options  are  identified 
or  old  options  refined. 

Include  as  much  detail  as 
possible  in  the  alternative 


descriptions.  As  a  minimum, 
provide  the  following: 

■  System  characteristics 
— size,  weight,  shape,  etc. 

■  System  performance  — 
rate  of  fire,  speed,  range, 
lethal  area,  payload,  detec¬ 
tion  range,  availability, 
etc.,  as  appropriate. 

■  Special  support  re¬ 
quirements  —  computer  re¬ 
sources,  maintenance,  fuel, 
supply,  etc. 

■  An  estimate  of  the  life 
cycle  cost  for  each  of  the 
alternatives  in  terms  of  R&D, 
investment,  O&S,  and  disposal. 

During  the  Phase  0  COEA 
effort,  there  will  be 
considerable  uncertainty  in 
both  the  performance  and  cost 
estimates.  In  many  cases,  they 
are  orily  predictable  in  terms 
of  error  or  confidence  bands. 
The  uncertainties  and  the 
source  of  all  information  must 
be  documented. 

ANALYSIS  -  Analysis 

includes  such  activities  as 
methodology  development,  ef¬ 
fectiveness  analysis,  life 
cycle  cost  analysis,  cost/ 
effectiveness  analysis,  trade¬ 
off  analysis,  and  other 
supporting  analyses  such  as 
logistics . 

METHODOLOGY  -  A  systematic  v 
methodology,  the  first  step  in 
the  analysis  process,  must  be 
developed  to  determine  how  the 
alternatives  will  be  eval¬ 
uated.  The  methodology  should 
be  clearly  defined  for  the 
effectiveness  analysis,  cost 
analysis,  and  any  required 
supporting  analysis.  In 
developing  the  methodology: 


■  Confirm  that  the 
methodology  is  logically 
sound . 

■  Make  sure  all  critical 
elements  are  included. 

■  Coordinate  the  method¬ 
ology  with  OASD(PA&E)  for  ACAT 
ID  programs  or  an  independent 
source  (usually  AFMC/OAS)  for 
other  acquisition  categories. 

EFFECTIVENESS  ANALYSIS 
The  effectiveness  analysis 
quantitatively  measures  if  the 
performance  level  achieved  by 
a  given  alternative  satisfies 
a  set  of  specific,  pre- 
established  requirements.  Key 
elements  of  the  effectiveness 
analysis  are  functional  objec¬ 
tives,  measures  of  effec¬ 
tiveness  (MOEs) ,  measures  of 
performance  (MOPs) ,  and  models 
and  data.  When  performing  the 
effectiveness  analysis: 

■  Identify  the  proba¬ 
bility  that  the  proposed 
system  will  operate  when 
required. 

■  Include  issues  of 
survivability  and  vulner¬ 
ability  to  countermeasures. 

■  Address  issues  of 
availability,  maintainability, 
and  safety. 

FUNCTIONAL  OBJECTIVES 
Functional  objectives  are 
goals  that  the  system  will  be 
expected  to  meet.  They  act  as 
standards  for  comparing  alter¬ 
natives.  Functional  objectives 
should: 

■  Translate  the 

requirements  (tasks  to  be 
accomplished)  of  the  MNS  and 
ORD. 

■  Be  measurable  in  terms 
of  quantitative  measures  of 
effectiveness  (MOEs) . 

■  Be  tailored  to  the  type 
of  system  under  consideration 
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(e.g. ,  the  new  weapon  system 
will  be  capable  of  attacking 
and  killing  xyz  targets) . 

■  Relate  to  the  mission 
need  (e.g.,  the  functional 
objective  of  destroying 
bridges  is  probably  not 
pertinent  if  the  mission  need 
is  to  locate  and  destroy 
aircraft  shelters) . 

■  Relate  to  system  design 
requirements  such  as  reli¬ 
ability,  availability,  main¬ 
tainability  (RAM) . 

MEASURES  OF  EFFECTIVENESS  - 
Be  careful  not  to  confuse 
measures  of  effectiveness 
(MOEs)  and  functional  objec¬ 
tives.  A  functional  objective 
is  a  goal;  an  MOE  is  the 
degree  to  which  the 
alternatives  satisfy  the  goal. 
An  MOE  for  the  above  example 
is  the  probability  of 
achieving  a  kill  on  the  xyz 
target.  When  developing  MOEs 
consider  the  following: 

■  MOEs  should  provide  a 
measure  of  how  well  the 
alternative  is  meeting  the 
functional  objectives. 

■  Try  to  identify  more 
than  one  MOE.  Increasing  the 
number  of  MOEs  will  tend  to 
increase  confidence  in  the 
robustness  of  the  results. 

■  The  set  of  MOEs 
developed  should  encompass  all 
of  the  functional  objectives. 

MEASURES  OF  PERFORMANCE  - 
Measures  of  performance  (MOPs) 
are  defined  as  quantitative 
measures  of  the  lowest  level 
of  physical  performance 
(range,  velocity,  throughtput, 
etc.)  or  physical 
characteristics  (height, 
weight,  volume,  frequency, 
etc.)  of  an  alternative.  In 
short ,  MOPs  are  system 
specifications.  MOPs  such  as 


range  and  speed  or  height  and 
volume  should  relate  to  MOEs 
such  that  the  effects  of  a 
change  in  the  MOP  can  be 
related  to  a  change  in  the 
MOE.  MOPs  should  indicate 
exactly  what  system  per¬ 
formance  characteristic  is  to 
be  measured.  When  developing 
MOPs : 

■  Use  such  specification 
items  as  range,  speed,  pay- 
load,  and  radar  cross  section 
as  MOPs. 

■  Identify  character¬ 
istics  such  as  height,  weight, 
and  volume  that  result  from  a 
system  design  solution. 

■  MOPs  should  provide 
information  on  whether  the 
system  has  met  required 
specifications. 

MODELS  AND  DATA  -  Accred¬ 
ited  models  and  data  are 
essential  to  the  effectiveness 
analysis  process.  The  choice 
of  models  will  be  driven  by 
the  need,  operational  concept, 
threat,  functional  objectives, 
MOEs,  and  MOPs.  Each  model 
should  be  sufficiently 
detailed  so  that  it  can 
accommodate  the  chosen 
scenario(s) ,  as  well  as  assess 
the  sensitivity  of  the  alter¬ 
natives  to  changes  in  assump¬ 
tions,  threat,  etc.  When 
choosing  the  model(s),  con¬ 
sider  the  following; 

■  If  possible,  select 
models  and  data  that  have  been 
screened  through  a  formal 
Verification,  Validation,  and 
Accreditation  process  (W&A)  . 

■  Choose  the  simplest 
model  (s)  that  can  perform  the 
necessary  calculations  and  the 
appropriate  sensitivity  in¬ 
vestigations  . 
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■  Use  existing  widely 
accepted  models  where  pos¬ 
sible.  Development  of  large 
scale  models  can  be  expensive 
and  time  consuming. 

■  Apply  the  model  to  the 
baseline  alternative  first.  In 
general,  the  existing  system 
will  be  better  understood  than 
any  conceptual  alternative. 
Explain  any  differences 
between  model  results  and 
known  baseline  performance. 

■  Make  sure  the  model  is 
well  understood  and  that  any 
apparent  anomalies  can  be 
explained. 

■  Where  possible,  apply 
the  same  model  to  all 
alternatives. 

An  important  point  to 
remember  is  that  data  should 
undergo  many  of  the  same 
checks  performed  on  models. 

LIFE  CYCLE  COST  ANALYSIS  - 
Life  cycle  cost  analysis 
provides  a  systematic  pro¬ 
cedure  for  estimating  the 
aggregate  cost  of  alternative 
systems,  and  it  allows  a 
comparison  between  alter¬ 
natives  based  on  economy. 
System  life  cycle  cost 
consists  of  acquisition  (R&D 
and  investment) ,  operating  and 
support,  disposal,  and 
military  construction  (MILCON) 
costs.  When  performing  a  life 
cycle  cost  analysis: 

■  Consider  acquisition 
costs,  operations  and  support 
costs,  and  disposal  costs  for 
each  alternative. 

■  Determine  if  indepen¬ 
dent  cost  estimates  can  be 
used  for  comparing  alter¬ 
natives. 

ACQUISITION  COST  -  This 
term  is  used  within  DOD  to 
denote  the  aggregation  of 


costs  to  develop,  produce,  and 
deploy  a  weapon  system  in  its 
operational  environment.  It 
includes  R&D  and  investment 
costs.  Acquisition  cost 
commences  with  Phase  0  and  is 
usually  completed  when  the 
last  production  unit  is 
delivered  to  the  using 
command.  When  developing  the 
acquisition  cost: 

■  Consider  R&D  and 
investment  cost  for  each 
alternative. 

■  Take  all  computer 
resources,  special  support 
equipment,  and  facilities  into 
account . 

OPERATION  AND  SUPPORT  (O&S) 
COST  -  O&S  costs  are  those 
program  costs  necessary  to 
operate,  maintain,  and  support 
system  capability.  These  costs 
include  personnel,  operations 
and  maintenance,  and  recurring 
procurement  appropriation 
costs  such  as  replenishment 
spares.  For  O&S  cost  develop¬ 
ment: 

■  Make  sure  the  CONOPS 
has  been  reviewed. 

■  If  available,  review 
related  integrated  logistics 
support  plans  for  essential 
information. 

DISPOSAL  COST  -  This  is  the 
cost  associated  with  getting 
rid  of  excess  or  surplus 
property  or  material  from  the 
Services'  inventories  under 
proper  authority.  It  may  also 
reflect  the  costs  of  hazardous 
waste  disposition  and  environ¬ 
mental  cleanup.  When  deter¬ 
mining  disposal  cost: 

■  Account  for  the  total 
system.  This  includes  support 
items  necessary  for  oper- 


at ions,  maintenance,  security, 
storage,  etc. 

MILCON  COST  -  This  is  the 
cost  of  supplies,  equipment, 
and  material  required  to 

provide  military  facilities 
necessary  to  accommodate  each 
of  the  alternatives.  When 
developing  MILCON  costs: 

■  Make  sure  MILCON 

requirements  have  been 

identified  for  all  alter¬ 
natives  . 

MODELS  AND  DATA  -  As  with 
effectiveness  models  and  data, 
cost  models  and  data  are 
critical  to  the  performance  of 
the  COEA  function.  When 
selecting  these  items: 

■  Make  sure  the  cost 
models  and  data  selected  to 
support  the  COEA  are 
accredited  by  the  responsible 
costing  agency. 

COST/EFFECTIVENESS  ANALYSIS 
A  cost/effectiveness 

analysis  is  an  analytical 
method  examining  the  cost  and 
performance  of  alternative 
means  of  accomplishing  desired 
military  missions.  It  requires 
the  definition  of  objectives, 
as  well  as  identification  of 
alternative  ways  of  achieving 
each  objective.  When  per¬ 
forming  the  cost/effectiveness 
analysis : 

■  Identify  alternatives 
meeting  system  objectives  that 
yield  the  greatest  benefit  for 
a  given  cost  or  the  required 
level  of  benefit  at  the  lu  2st 
cost. 

■  Maintain  a  balance  be¬ 
tween  cost  and  effectiveness. 

■  Compare  equal  cost  or 
equal  effectiveness  alter¬ 
natives. 


■  Stay  away  from  schemes 
with  weighted  measures. 

■  Avoid  ratios:  they  tend 
to  ignore  sufficiency  and  mask 
important  diif'f erences . 

TRADEOFF  ANALYSIS  -  The 
purpose  of  the  tradeoff 
analysis  is  to  identify  the 
strengths  and  weaknesses  of 
the  alternatives  and  to 
identify  the  preferred  solu¬ 
tion.  The  MOPs  and  the  cost 
for  each  alternative  are  used 
to  investigate  the  utility  of 
changes  in  each  measure.  This 
process  is  one  of  comparing 
alternatives  on  the  basis  of 
equal  cost  and/or  equal 
effectiveness,  assessing  the 
effects  of  uncertainty  and 
sensitivity,  and  establishing 
performance  and  cost  thresh¬ 
olds.  The  final  task  is  to 
prioritize  the  alternatives. 
There  is  no  cookbook  approach. 
Key  items  should  be  assessed, 
such  as  cost,  effectiveness, 
logistics,  risk,  counter¬ 
measures,  and  military  util¬ 
ity.  When  executing  the 
tradeoff  analysis: 

■  Compare  alternatives  on 
the  basis  of  equal  cost  and/or 
equal  effectiveness. 

■  Be  concerned  with  real 
issues,  such  as  national 
security,  force  structure,  the 
world  political  scene,  U.S. 
industry,  the  environment,  and 
energy  needs. 

OTHER  SUPPORTING  ANALYSES 
-  This  category  of  analysis 
includes  all  other  applicable 
analyses  that  do  not  fall 
under  the  categories  of  cost 
or  effectiveness  analysis. 
Support  analyses  should 
identify  any  analyses  needed 
to  support  the  COEA,  such  as 
MNAs,  MAAs,  environmental 
analyses,  economic  analyses. 
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logistics  analyses,  and 
manpower  studies. 

ORGANIZATIONAL 

RESPONSIBILITIES 

As  stated  earlier,  it  is 
the  responsibility  of  the 
operating  command  to  prepare 
the  COEA.  The  operating 
command  is  therefore  obligated 
to  develop  its  own  organ¬ 
ization  and  procedures  for 
accomplishing  the  COEA.  The 
following  material  is  sug¬ 
gested  as  a  model  for  the 
operational  organization.  It 
should  be  tailored  to  the 
specific  operating  command's 
organization  and  approach  to 
weapon  systems  acquisition. 

ORGANIZATION  -  At  least  two 
groups  should  be  established 
when  developing  COEAs  for  a 
major  program:  a  study  team 
and  a  study  advisory  group 
(SAG) .  The  suggested  compos¬ 
ition  and  roles  of  these  two 
groups  are  discussed  in  the 
following  sections. 

STUDY  TEAM  -  The  study  team 
should  include  members  from 
the  operating  command's  DCS 


for  Operations,  Requirements, 
Plans,  Intelligence,  Financial 
Management/Comptroller,  and 
Logistics,  as  well  as  repre¬ 
sentatives  from  other  func¬ 
tional  areas  as  needed.  For 
example,  a  weather  staff 
officer  is  important  for  COEAs 
with  environmental  sensitiv¬ 
ities.  The  study  team  should 
also  include  members  from  the 
implementing  command  (usually 
AFMC)  and  support  agencies 
(AFOTEC,  AFSAA,  AFCAA,  AFMC, 
etc.).  On  programs  involving 
AFMC,  members  from  the 
appropriate  AFMC/XR  and  FM 
centers  should  be  included.  On 
joint  programs,  members  from 
other  services  or  the  Unified 
Commands  should  be  included. 
Study  team  members  should  be 
responsible  for  material  to  be 
supplied  by  their  respective 
DCS.  The  study  team  leader 
should  exercise  overall 
responsibility  for  coord¬ 
ination  of  the  efforts.  He  or 
she  should  report  directly  to 
the  commander  (CC)  of  the 
operating  command. 

Figure  6  illustrates  the 
relationship  between  the  study 
team  and  the  supporting  organ- 


Figure  6.  Relationship  Between  Study  Team 
and  Supporting  Organizations 
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izations.  The  study  team  will 
concentrate  on  two  general 
areas:  cost  estimates  and 

operational  effectiveness. 

The  operating  command  study 
team  member  from  FM  should  be 
responsible  for  coordinating 
the  cost  estimates.  Operating 
command  support  should  be 
drawn  from  the  cost 
communities  of  the  functional 
areas  (engineering,  personnel, 
logistics,  etc.).  Support 
should  also  be  drawn  from  the 
FM  organizations  within  the 
implementing  command. 

The  operational  effective¬ 
ness  area  should  be  coordin¬ 
ated  by  a  team  member  from  the 
operating  command  staff, 
generally  from  the  DCS  for 
Operations  or  Requirements.  As 
with  the  cost  estimates, 
support  should  be  drawn  from 
the  operational  command 
functional  areas,  as  required, 
and  from  the  appropriate 
functional  areas  of  the 
implementing  command.  A 
suggested  division  of 

responsibilities  between  the 
study  team  and  the  supporting 
organizations  is  given  in 
Figure  7. 


STUDY  ADVISORY  GROUP  -  In 
order  to  ensure  that  the 
operating  and  implementing 
commands  and  the  decision 
makers  continue  in  concert 
throughout  the  development,  a 
study  advisory  group  (SAG) 
should  be  formed.  The  SAG 
should  consist  of  senior 
representatives  (0-6  or  above) 
from  the  principal  organ¬ 
izations  on  the  operating 
command  staff  and  repre¬ 
sentatives  from  the  imple¬ 
menting  command,  SAF/AQ, 
SAF/FM,  and  USAF/XO.  For  ACAT 
I  programs,  the  SAG  should 
include  a  representative  from 
the  Office  of  the  Assistant 
Secretary  of  Defense  for  Pro¬ 
gram  Analysis  and  Evaluation 
(OASD/PA&E) . 

The  SAG  should  review  the 
COEA  effort  at  the  following 
specific  points  in  the  COEA 
development  process: 

■  Completion  of  the  COEA 
plan  prior  to  the  Phase  0 
analysis. 

■  Completion  of  the  Phase 
0  COEA  analysis  prior  to 
submitting  results  to  the 
Milestone  I  review. 
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Figure  7.  Suggested  Division  of  Responsibility 
Between  Study  Team  and  Supporting  Organizations 


■  Completion  of  the 
updated  COEA  plan  prior  to  the 
Phase  I  analysis. 

■  Completion  of  the  Phase 
I  COEA  analysis  prior  to 
submitting  results  to  the 
Milestone  II  review. 


For  programs  requiring  a 
significant,  long-term  anal¬ 
ytical  effort,  the  SAG  should 
conduct  a  review  at  about  the 
midpoint  of  the  analysis.  A 
review  of  the  alternatives 
selected  for  inclusion  in  the 
analysis  also  may  be  included 
for  major  efforts. 


REPORT  FORMAT 


The  COEA  report  should  be 
as  brief  as  possible  (100 
pages  maximum) .  It  should 
summarize  the  major  elements 
of  the  COEA  and  present  the 
results  of  the  analysis  in 
terms  of  relative  afford¬ 
ability  and  utility  of  alter¬ 
natives.  A  suggested  report 
format  is  presented  in  Figure 
8. 
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Figure  8.  Suggested  COEA 
Report  Format 


REVIEW  PROCESS 

Experienced,  senior-level 
people  in  the  coordination 
chain  should  review  the  COEA 
at  specified  stages  in  its 
development.  Their  job  is  to 
ensure  focused,  sustained 
progress  in  developing  a 
useful  analysis  to  support 
program  decisions  by  the 
Acquisition  Executive  and/or 
the  Defense  Acquisition  Board 
(DAB) .  The  coordination  pro¬ 
cess  offers  a  unique  oppor¬ 
tunity  to  improve  the  quality 
and  develop  wide  support  for 
the  COEA.  Early  coordination 
is  crucial  to  avoid  dis¬ 
connects  and  revision  late  in 
the  COEA  development  process. 
While  there  is  no  formal 
checklist  to  assure  a  suc¬ 
cessful  COEA,  COEAs  are 
evaluated  on  many  of  the 
following  points; 

■  What  is  the  ^-roblem 
being  addressed?  Is  it  the 
real  problem  or  only  a  symptom 
of  a  more  basic  concern? 

■  Is  the  scenario 
consistent  with  the  Defense 
Planning  Guidance?  Is  the 
threat  consistent  with  the  DIA 
estimate? 

■  Are  the  assumptions  and 
constraints  explicitly  ident¬ 
ified?  Are  they  really  con¬ 
straints?  Are  they  unduly 
restrictive?  Are  they  defen¬ 
sible? 

■  Is  the  CONORS 
reasonable  and  complete? 

■  Have  any  feasible, 
significant  alternatives  been 
omitted? 

■  Were  multiple  MOEs 
used?  Do  they  relate  to 
functional  objectives?  Are  the 
real  measures  of  effectiveness 
or  only  measures  of  per¬ 
formance? 
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■  Have  all  relevant  costs 
been  identified?  Have  signif¬ 
icant  cost  drivers  been 
identified?  Have  costs  been 
reviewed  by  an  independent 
second  source? 

■  Are  the  models  used  in 
the  evaluation  adequately 
explained?  Have  they  been 
through  a  W&A  process?  Are 
the  results  obtained  for  the 
baseline  system  consistent 
with  existing  data  from 
previous  tests  and  studies? 

■  Has  the  database  for 
the  analysis  been  validated 
through  engineering  analysis 
or  test  information? 

■  Does  the  COEA  address 
the  cost  and  effectiveness  of 
all  the  proposed  alternatives? 

■  Are  the  criteria  for 
selecting  or  ranking  preferred 
alternatives  identified  ex¬ 
plicitly?  Are  they  meaningful 
and  intuitively  acceptable? 

■  Does  the  COEA  identify 
the  preferred  alternative (s) ? 
Are  the  reasons  explicitly 
stated?  Does  the  preferred 
alternative (s)  offer  distinct 
advantages  over  those  alter¬ 
natives  not  selected? 

■  Are  the  results  of  the 
analysis  intuitively  satis¬ 
fying?  Are  they  convincingly 
substantiated? 

■  Were  sensitivity  anal¬ 
yses  conducted  to  show  how 
changes  in  assumptions  about 
performance,  threat,  environ¬ 
ment,  alternative  costs,  etc. 
affect  utility,  cost  effect¬ 
iveness,  or  schedule? 

■  Were  other  supporting 
analyses  such  as  logistics, 
environmental  impact,  energy, 
economic  and/or  manpower 
studies  performed? 

■  Has  the  risk  associated 
with  the  development  of  each 
alternative  been  adequately 
addressed? 


■  Was  the  report  format 
followed?  If  an  item  was  not 
included,  is  there  justif¬ 
ication  for  its  omission? 

SUMMARY 

The  elements  of  a  COEA 
range  across  several  areas, 
such  as  threat,  mission  needs 
and  U.S.  capabilities,  inter¬ 
relationship  of  systems, 
multi-role  systems,  measures 
of  effectiveness,  measures  of 
performance,  supportability , 
costs,  and  cost-effectiveness 
comparisons.  A  successful  COEA 
depends  upon  understanding  and 
applying  the  elements  effect¬ 
ively  in  the  DOD  system 
acquisition  process. 

Today's  military  system 
development  programs  generally 
exist  in  a  complex  environ¬ 
ment,  utilizing  advanced 
science  and  engineering  tech¬ 
nologies.  Funding  shortages, 
competition  within  DOD  for 
program  dollars,  the  overall 
reduction  of  resources,  and 
the  opportunity  for  technical 
mistakes  places  a  greater 
emphasis  on  the  COEA  process. 
As  a  result,  it  is  strongly 
recommended  that  the  elements 
of  the  COEA  be  identified  and 
understood  and  that  a  study 
plan  be  initiated  as  early  as 
possible  in  the  acquisition 
process.  The  ideal  time  to 
start  the  plan  is  with  the 
establishment  of  the  require¬ 
ments  definition.  The  analysis 
that  accompanies  the  develop¬ 
ment  of  the  requirements  may 
very  well  be  the  first 
evaluation  of  operational 
effectiveness.  Understanding 
the  requirements  is  the  key  to 
successful  system  development 
via  the  COEA  process. 
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NOTE 

A  more  detailed  discussion 
of  the  COEA  process  is 
contained  in  AFMC  Pamphlet 
173-1,  AFMC  Cost  &  Operational 
Effectiveness  Analysis  (COEA) 
Guide,  dated  December  1992. 
Included  in  this  document  are 
attachments  on  DOD  Acquisition 
Categories,  Key  Elements  of  a 
COEA,  Sources  of  Costing 
Support,  and  Sources  of 
Performance  Simulation  Sup¬ 
port.  The  key  elements  of  the 
COEA  follow  the  form  of  the 
DOD  publication,  Draft  Cost 
and  Operational  Effectiveness 
Analysis  (COEA)  Guidelines , 
Office  of  the  Assistant 
Secretary  of  Defense  (Program 
Analysis  and  Evaluation) , 
February  1990. 
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EFFECTS  OF  THE  POLITICAL  ENVIRONMENT  ON  IMPLEMENTATION  OF 
TECHNOLOGICALLY  ADVANCED  WEAPON  SYSTEMS: 

The  North  American  B-70  Bomber 
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ABSTRACT 


This  paper  discusses  bringing  technologically  advanced  weapon 
systems  into  current  military  inventory,  and  why  difficulty  is 
often  encountered  in  the  process.  This  paper  uses  one  example,  the 
North  American  B-70  bomber  program,  to  illustrate  inconsistent 
political  support  and  the  reasons  behind  it.  Like  many  other 
advanced  programs,  the  B-70  was  cancelled  due  to  a  lack  of 
political  support.  The  research  associated  with  this  paper 
compiled  a  complete  and  chronological  history  of  the  political 
actions  surrounding  the  North  American  B-70  bomber  program,  from 
its  beginnings  to  the  rollout  of  the  first  prototype,  in  May  1964. 
This  paper  outlines  what  elements  define  the  political  environment 
a  system  must  survive,  and  uses  the  B-70  bomber  program  as  an 
example. 
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INTRODUCTION 

Implementing  large-scale  technology 
advancements  into  current  military  inventory 
is  often  controversial  and  costly.  The  B-2 
bomber,  F-22  advanced  tactical  aircraft. 
Strategic  Defense  Initiative,  LHX  helicopter, 
and  V-22  Osprey  are  recent  examples.  Like 
their  predecessors,  these  programs  will  only 
survive  if  the  political  climate  supports  their 
procurement.  The  difficulty  in  bridging  the 
gap  between  technological  jumps  and 
implementation  into  the  operational  field  is 
this  often  changing  political  climate. 

Two  basic  questions  determine  this 
climate:  Will  the  new  technology  contribute 
enough  to  justify  its  cost?  And  is  there  a 
more  worthy  technology  competing  for  the 
same  dollars?  Complications  arise  when 
individual  perspectives,  viewpoints  of  key 
decision-makers,  change  during  the  course  of 
the  program. 

The  difficulty  as.sociated  with 
implementing  aggressive  technology 
advancements  into  military  inventory  is 
illustrated  by  the  attempts  to  acquire  the 
North  American  B-70  bomber.  This  30- 
year-old  story  shows  how  political  climate, 
regardless  of  strategic  mission  requirements 
or  technical  maturation,  deteimines  the  fate 
of  innovative  advancements  in  weapons 
systems.  In  the  case  of  the  B-70, 
inconsistent  political  support  kept  funding  up 
and  down  for  years,  eventually  killing  the 
program. 

By  comparison,  the  B-2  bomber 
controversy  almo.st  pales.  The  B-70,  and  its 
derivative  the  RS-70,  "...precipitated  a  fight 
so  intense  as  to  lead  to  a  Constitutional 


crisis"  (34:244).  With  billions  of  dollars  at 
stake,  the  President  and  Congress  were  at 
odds  year  after  year,  finally  resolving  the 
dispute  and  dooming  the  program  in  a  casual 
meeting  in  the  White  House  Rose  Garden. 

DISCUSSION 

Overview  and  Concept 

The  North  American  B-70  was 
"perhaps  the  most  imaginative  and 
sophisticated  airplane  ever  designed"  (42:30). 
Intended  to  replace  the  B-52  as  the  primary 
U.S.  bomber,  the  B-70  was  designed  to  fly 
continuously  at  Mach  3,  at  altitudes  over 
70,000  feet,  and  attack  predetermined  targets 
with  nuclear  bombs.  The  B-70  represented 
over  1000  patents  and  thousands  of  technical 
innovations.  "She  is  so  unlike  previous 
aircraft  that  comparisons  are  almo.st 
meaningless,"  wrote  Hunter  (21:157). 

The  B-70’s  costs  were  as  staggering 
as  its  performance.  The  average  cost  of  a 
single  B-70  (based  on  a  250  plane  program) 
would  be  over  $24.5  million,  more  than 
three  times  the  cost  of  a  B-52  it  was 
replacing.  Engineering  and  design  costs 
were  estimated  at  $1.3  billion. 

The  lifespan  of  the  B-52  was  not 
envisioned  much  beyond  the  late  1960s,  and 
in  the  early  1950s,  the  Air  Force  began 
looking  for  a  new,  long-range  bomber  to 
become  the  nation’s  next  strategic 
centerpiece.  Progress  seemed  to  indicate  this 
next  bomber  would  be  a  large,  high  flying 
supersonic  aircraft,  and  certainly  brought 
into  the  strategic  inventory  as  soon  as 
technology  permitted. 
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Project  Beginnings 

In  1954,  Air  Force  General  Curtis 
E.  LeMay,  commander  of  the  Strategic  Air 
Command,  officially  defined  mission 
requirements  for  a  new,  advanced  jet 
bomber.  His  objective  was  a  plane 
combining  the  range  and  payload  of  the  B- 
52  with  the  supersonic  speed  of  the  B-58. 
The  first  design  proposals  did  not  meet 
LeMay’s  requirements  and  the  contractors 
went  back  to  the  drawing  boards. 

In  March  1956,  a  secret  research 
paper  by  A.J.  Eggers  and  C.A.  Syvertson 
described  an  aerodynamic  phenomenon 
called  "compression  lift,"  allowing  a 
supersonic  vehicle  to  increase  lift  by  riding 
its  own  shock  wave.  This  breakthrough 
made  it  both  feasible  and  practical  to  fly  at 
supersonic  speeds  for  the  entire  mission,  not 
just  "dash"  through  the  target  area.  North 
American  Aviation  discovered  this  paper 
during  a  routine  periodical  search  and 
Eggers  and  Syvertson  then  collaborated  with 
North  American  engineers  and  designers. 
Referring  to  compression  lift.  Dr  Hugh  L. 
Dryden  stated  "...a  strange  and  wonderful 
thing  happened.  It  was  as  if  the  pieces  of  a 
jig-saw  puzzle  began  falling  into  place." 
(52:5) 

In  December  1957,  North 
American’s  design  for  a  Mach  3 
intercontinental  bomber  (featuring 
compression  lift)  was  selected  over  Boeing’s. 
Boeing,  whose  B-17,  B-29,  B-47,  and  B-52 
made  them  the  recognized  bomber  experts, 
filed  a  protest  over  the  decision.  A  special 
Congressional  inquiry,  however,  found  the 
decision  sound.  The  principle  factor  of  the 
decision  to  select  North  American’s  design 
was  the  com-pression  lift  feature. 

One  month  after  the  .selection,  a 
contract  was  signed  with  North  American  for 
12  test  aircraft,  plus  a  delivery  of  fifty 
bombers  to  SAC.  The  Air  Force  was 
planning  for  an  eventual  strength  of  250  of 
the  new  bombers. 


1958 

In  February  1958,  the  project  was 
officially  designated  the  B-70,  with  first 
flight  due  in  Dec  1961,  and  SAC  deliveries 
scheduled  for  1964. 

When  the  Eisenhower  Administration 
cancelled  the  nuclear-powered  aircraft 
program  in  1958,  the  B-70  became  the  Air 
Force’s  only  new  bomber  program.  The  Air 
Force  therefore  increased  the  priority  of  the 
program  and  accelerated  its  schedule  by 
eighteen  months. 

Also  that  year,  the  Department  of 
Defense  Reorganization  Act  of  1958  was 
passed.  This  authorized  the  Secretary  of 
Defense  to  "run  the  Pentagon."  The 
Secretary  could  now  assign  the  development, 
production,  and  operational  use  of  weapon 
systems  for  any  of  the  service  branches. 
However,  by  custom,  the  services  still 
dominated  the  policy  and  decision  making  in 
the  significant  areas.  This  legislation 
provided  the  groundwork  for  expanding  the 
role  of  the  Defense  Secretary  in  the 
management  of  defense  acquisition 
programs. 

1959 

In  1959  the  B-70  first  came  under 
fire,  and  received  a  setback  from  which  it 
would  never  recover. 

President  Eisenhower  aimed  at 
remaining  militarily  ahead  of  the  Russians, 
but  at  the  same  time,  operating  within 
budgets  that  reflected  the  nation’s  economic 
status.  Strategically,  Eisenhower’s  position 
was  to  rely  mainly  on  manned  bombers,  and 
remain  cautious  concerning  intercontinental 
missile  deployment.  Even  as  missile 
technology  matured,  the  country  should 
maintain  a  "mix"  of  strategic  systems,  both 
bombers  and  ICBMs  .  But  the  ICBM  was 
developing  faster  than  expected  ,  and  the 
Defense  Department  found  their  ICBM 
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programs  eating  up  such  huge  sums  of 
money  that  new  manned  aircraft  programs 
had  to  be  critically  reappraised. 

In  December  1959,  a  special 
Presidential  study  questioned  the  utility  of 
the  manned  bomber  compared  to  the  ICBM. 
Since  the  U.S.S.R.  was  developing  its  own 
ICBMs,  some  considered  the  manned 
bomber  obsolete;  the  Air  Force  could  not 
possibly  launch  its  fleet  of  bombers  in  time 
to  avoid  destruction  by  missile.  Also,  the 
U.S.  ICBM  programs  had  been  making  great 
progress,  while  the  B-70,  "...trying  to 
revolutionize  the  bomber  in  one  mighty  leap, 
ran  into  predictable  techno-logical  snags" 
(35:32).  Furthermore,  the  Cold  War  was 
experiencing  a  "thawing,"  marked  by  Soviet 
Premier  Khrushchev’s  visit  to  the  United 
States. 

Realizing  the  country  could  not 
support  every  multi-billion  dollar  defense 
program  in  sight  and  stay  solvent,  the 
Administration  made  moves  to  terminate  the 
B-70  program.  The  FY  1960  budget  was 
trimmed  by  $25  million,  and  the  FY  1961 
budget  was  slashed  from  $365  million  to 
only  $75  million.  This  resulted  in  a  trimmed 
and  stretched  out  program,  where  only  two 
experimental  airplanes  would  be  built.  One 
airplane  would  be  a  prototype,  designated 
XB-70,  stripped  of  all  complex  bombing, 
navigation,  and  radar  subsystems,  and  flight 
tested  a  year  later  than  scheduled.  The 
second  aircraft,  designated  YB-70  and 
upgraded  with  the  navigation  and  bombing 
equipment,  would  be  flown  a  year  later. 

This  move  created  an  uproar  among 
bomber  proponents,  who  now  saw  only  a 
bleak  future  for  the  program.  Although 
officially  only  "trimmed  and  stretched  out," 
many  feared  the  program  restructure  would 
eventually  kill  the  B-70  altogether.  They 
believed  when  a  B-70  prototype  flew  without 
vital  subsystems  the  achievement  would 
be  "as  hollow  as  its  empty  airframe,"  giving 
critics  even  more  grounds  to  abolish  the 


program  because  it  obviously  has  not 
produced  a  truly  useful  vehicle.  (19:21) 
Robert  Hotz,  editor  of  Aviation  Week,  called 
this  move  "...one  of  the  most  dangerous 
decisions  made  in  this  country  during  the 
past  decade"  (19:21). 

Questions  were  raised  regarding  the 
overall  future  of  the  manned  bomber.  There 
were  no  longer  any  new  programs  on  the 
horizon.  The  production  lines  for  the  B-52 
and  B-58  were  scheduled  to  shut  down 
altogether  in  1962.  Also,  the  Martin 
Company,  proven  and  successful  bomber 
manufacturers,  abandoned  manufacturing 
aircraft  altogether  . 

President  Eisenhower’s  cutback  of  the 
B-70  seemed  to  indicate  a  decision  to 
eventually  replace  all  manned  strategic 
aircraft  with  ballistic  missiles.  As 
Ei.senhower  moved  to  make  ICBM 
development  the  country’s  highest  defense 
priority,  the  Air  Force’s  highest  priority 
programs  were  still  additional  B-52s  and  the 
B-70.  The  Air  Force  was  reluctant  to  adopt 
a  missile  based  strategy  because  "the  bomber 
represented  the  heart  of  the  Air  Force’s 
guiding  military  doctrine,  the  overriding 
importance  of  strategic  air  power"  (35:32). 

At  the  Pentagon,  in  the  Air  Force, 
and  in  Congress,  lines  were  drawn  between 
advocates  of  manned  bombers  and  missiles. 

1960 

The  B-70  quickly  developed  into  one 
of  the  most  controversial  weapons  issues 
ever.  Legislators,  who  had  substantial 
aviation  industry  in  their  home  states,  led  in 
the  attack  upon  the  B-70  cutbacks  . 

The  politics  now  commonly 
as.sociated  with  large  scale  defense  programs 
were  just  emerging  in  1960,  as  the  size  of 
military  projects  was  increasing.  Replacing 
the  several  hundred  B-52s  with  B-70s 
promised  to  be  at  least  a  $6  billion  effort. 

In  1959,  Congress  passed  Public  Law 
86-149,  which  increased  their  control  over 
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acquisition  of  large  defense  programs. 
Congress  passed  additional  legislation  in 
1960  which  gave  the  House  and  Senate 
Anned  Services  Committees  (HASC  and 
SASC)  increased  influence  in  the  annual 
defense  budget.  Prior  to  1960,  the 
Committees  only  authorized  funding  for 
military  construction  projects.  But 
recognizing  the  growing  size  of 
appropriations  for  ships,  aircraft,  and 
missiles,  the  Committee  members  succeeded 
in  working  legislation  that  permitted  them 
authorization  of  these  programs  as  well. 
"With  contracts  involving  billions  of  dollars, 
the  fate  of  entire  companies,  the  economic 
welfare  of  communities,  and  the  careers  of 
ambitious  politicians  all  hinged  on  winning 
key  defense  projects."  (35:50) 

Early  in  1960,  the  Senate  conducted 
its  own  investigation  into  the  need  for  the 
new  bomber,  chaired  by  Senator  Lyndon 
Johnson.  General  Thomas  S.  White,  Air 
Force  Chief  of  Staff,  testified  against  the 
Administration’s  cutback  of  the  program. 
Air  Force  General  Nathan  Twining, 
Chairman  of  the  Joint  Chiefs  of  Staff, 
testified  that  if  the  B-70  was  not  procured, 
the  United  States  would  have  no  bombers  in 
1967,  due  to  rapid  advances  in  bomber 
technology  making  the  B-52  and  B-58 
obsolete.  General  LeMay  argued  that 
ballistic  missiles  could  not  serve  all  the 
needs  of  deterrence  and  security.  Rather,  he 
believed  manned  bombers  together  with 
ICBMs,  would  be  necessary.  "Our  problem 
is  not  one  of  killing  our  opponents,  LeMay 
testified,  "Our  problem  is  one  to  keep  our 
opponents  from  killing  Americans  and  our 
allies....  If  we  just  want  to  kill  our 
opponents,  that  is  a  very  simple  problem.  It 
calls  for  a  lot  less  weapons  ...."(58:132) 

The  ability  of  an  aircraft  to  even 
penetrate  Soviet  airspace  fell  under  .scrutiny 
in  mid  1960.  During  the  late  1950s, 
Americans  believed  the  Soviets  had  no 
surface-to-air  missile  (SAM)  nor  interceptor 


aircraft  capable  of  reaching  an  aircraft  at 
80,000  ft  altitud.  However,  on  May  1,  1960, 
Soviet  SAMs  brought  down  a  U-2 
reconnaissance  aircraft  overflying  the  USSR, 
while  it  was  near  this  altitude.  The 
celebrated  Francis  Gary  Powers  U-2  "spy 
incident"  caused  a  reevaluation  of  the  B-70’s 
vulnerability  to  radar  detection  and  SAMs. 
Critics  argued  that  the  airplane’s  skin 
friction  at  Mach  3  would  make  an  easy 
target  for  a  heat  seeking  missile,  and  the 
bomber’s  size  would  be  easily  picked  up  on 
radar.  Some  foresaw  an  impenetrable  Soviet 
Union  by  the  years  the  B-70  was  scheduled 
to  become  operational.  But  advocates  of  the 
B-70  used  the  Soviet  defenses  for  their  own 
arguments.  Experts  estimated  it  would  cost 
over  $40  billion  for  the  Russians  to  upgrade 
their  defenses  to  counter  the  $10  billion  B- 
70  .  Many  agreed  with  Ed  Rees,  who  wrote, 
"If  the  B-70  accomplished  nothing  else  but 
obligated  the  Soviets  to  a  $40  billion  defense 
expenditure,  an  expenditure  that  would  not 
increase  her  offensive  strength  against  the 
United  States  homeland  by  one  warhead,  it 
will  have  proved  a  .superb  tradeoff."  (54:61) 

In  July  1960,  Johnson’s  Preparedness 
Investigating  Sub-Committee  of  the  Senate 
Armed  Services  Committee  released  their 
findings.  They  concluded  there  was  a  vital 
necessity  for  a  new  intercontinental  bomber, 
despite  ICBM  development.  Furthermore, 
the  nation  possessed  the  resources  and 
technology  necessary  to  build  the  B  -  7  0  . 
Finally,  experience  showed  that  stretching 
out  a  program  only  increased  costs  and 
wa.sted  valuable  time. 

With  the  B-70  need  reaffirmed. 
Congress  restored  its  funds,  and  also  voted 
$184  million  more  than  the  President’s 
budget  called  for.  Although  Congress 
authorized  additional  funds  for  the  plane, 
there  was  nothing  requiring  the  President  to 
so  spend  it.  "Congress  appropriates  military 
funds  based  on  DoD  requests,  and  then  the 
DoD  reallocates  as  it  wills,"  wrote  Pike 
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(47:1058).  This  allowed  the  Executive 
Branch  to  "impound,"  or  simply  not  spend, 
any  funds  it  considered  excessive.  (This  was 
not  the  first  time  this  issue  had  arisen.  In 
1949,  the  Truman  Administration  had 
impounded  $615  million  appropriated  by 
Congress  for  the  purchase  of  airplanes.) 
These  controversial  procurement  powers  of 
the  Legislative  vs.  the  Executive  Branches 
would  haunt  the  B-70  program  for  years  to 
come. 

Meanwhile,  as  Congress  was 
investigating  the  need  for  the  B-70,  the  issue 
had  also  become  an  important  item  in  the 

1960  Presidential  campaign.  John  F. 
Kennedy,  was  campaigning  in  California  in 
support  of  the  aiiplane,  which  would  provide 
jobs  at  the  North  American  manufacturing 
plant. 

Although  President  Eisenhower  was 
against  pursuing  the  B-70  as  a  full  weapons 
system,  he  released  $155  million  in 
additional  funds  the  very  day  before  the 
election.  His  reason  for  releasing  the 
money  "at  the  height  of  the  uproar"  was  that 
the  U.S.  was  taking  a  series  of  steps  to  cope 
with  the  Soviet  actions  following  the  May 
U-2  affair.  However,  others  saw  its  timing 
as  a  last  minute  attempt  to  win  California 
votes  for  Republican  candidate  Richard  M. 
Nixon. 

The  $155  million  again  restructured 
the  program  such  that  now  one  XB-70  and 
two  YB-70s  would  be  built,  followed  by 
twelve  fully  operational  B-70s. 

Although  the  B-70  had  played  an 
important  role  in  the  election,  with  the 
victorious  Democrats  strongly  advocating  the 
new  bomber,  within  a  month  of  taking 
office,  the  Kennedy  Administration  dropped 
its  plans  to  revive  the  B-70. 

1961 

In  January  1961,  as  pan  of  his  final 
State  of  the  Union  Message  to  the  Congress, 
outgoing  President  Eisenhower  stated,  "the 


bomber  gap  of  several  years  ago  was  always 
a  fiction..."  (67:14).  Secret  U-2 
reconnaissance  photographs  had  revealed  that 
the  U.S.  was  actually  far  ahead  in  bomber 
production.  As  the  new  Administration  was 
given  access  to  these  facts,  the  future  of  the 
B-70  program  once  again  vv'-is  in  doubt. 

Although  the  Department  of  Defense 
Reorganization  Act  of  1958  increased  the 
authority  of  the  Defen.se  Secretary,  it  was 
not  until  1961  that  the  was  fully  exercised. 
New  Secretary  of  Defense  Robert  S. 
McNamara  believed  in  active  management, 
and  with  the  authority  vested  in  the  office, 
he  played  a  major  role  in  weapon 
acquisition.s. 

The  Administration  compared  the 
cost  and  effectiveness  of  the  B-70  to  other 
weapons  systems.  McNamara  believed  in 
selecting  a  weapon  systems  by  "...dealing  not 
with  absolutes  but  with  comparatives" 
(58:40).  According  to  McNamara,  "We 
must  always  take  into  account  not  only  the 
planned  capabilities  of  the  proposed  weapon 
system,  but  also  its  full  cost  in  comparison 
to  the  cost  and  effectiveness  of  other  weapon 
systems  which  can  do  the  same  job,  perhaps 
in  somewhat  different  ways."  (58:40) 

McNamara  also  questioned  the  Air 
Force  position  that  the  B-52  would  be 
obsolete  by  the  late  1960s.  He  believed  the 
plane  could  be  used  into  the  1970s,  using 
stand-off  weapons,  thus  eliminating  the  need 
to  penetrate  Soviet  airspace.  (37) 

Furthermore,  McNamara  questioned 
the  technical  feasibility  of  the  airplane’s  role 
as  the  Air  Force  described  it.  At  80,000 
feet,  traveling  at  Mach  3,  the  B-70  would 
indeed  be  a  "manned  missile,"  incapable  of 
finding  mobile  or  hidden  targets;  limited  to 
predetermined  targets  of  known  position. 
The  B-70  appeared  to  lack  the  flexibility 
generally  attributed  to  manned  bombers. 
Finally,  McNamara  believed  the  high  flying, 
supersonic  B-70  would  be  more  easily 
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detected  by  Soviet  radar  than  the  current  B- 
52. 

In  March  1961,  President  Kennedy 
released  an  official  statement  that  America’s 
forthcoming  ICBM  capability  "...makes 
unnecessary  and  economically  unjustifiable 
the  development  of  the  B-70  as  a  full 
weapons  system...."  (46:20).  The 
Administration  received  heavy  criticism  for 
this  decision,  since  Kennedy  had  supported 
the  B-70  during  campaigning,  and  Vice- 
President  Johnson  had  chaired  the  Senate 
Investigation  that  had  urged  its  full 
production. 

Secretary  of  Defense  McNamara’s 
plan  for  the  B-70  now  reduced  it  to  four 
prototype  airplanes  designed  to  investigate 
problems  of  long-range  operations  at 
prolonged  supersonic  speed.  The 
Eisenhower  budget  for  FY  1962  was 
trimmed  to  $220  million,  less  than  half  the 
Air  Force  requirement. 

During  the  spring  of  1961,  the  Air 
Force  again  sought  to  reinstate  B-70  funding 
by  going  before  the  Congress.  Secretary  of 
the  Air  Force  Eugene  Zuckert  warned 
Congress  that  reliability  of  ICBM  systems 
had  not  yet  been  proven.  Defense  Secretary 
McNamara,  testified  he  was  against  any 
acceleration  of  the  B-70  program,  and  stated 
he  would  refuse  to  spend  any  additional 
funds  above  the  President’s  budget. 

The  House  and  Senate  Aimed 
Services  Committees  concurred  with  the 
Administration’s  plans  and  appropriated  only 
$220  million,  refusing  pleas  from  the  Air 
Force  to  increase  funding  and  develop  the  B- 
70  as  a  full  weapons  .system.  House  Armed 
Services  Committee  Chairman  Carl  Vinson 
agreed  that  the  President’s  plan  allowed  for 
an  orderly  development  of  the  aircraft  — 
providing  an  opportunity  to  test  the  technical 
feasibility  of  the  B-70  and  its  related 
subsystems  while  also  preserving  an  option 
of  full  production  if  it  proved  nece.s.sary. 
'When  the  defense  budget  went  to  the 


Appropriations  Committees,  for  the  first  time 
in  25  years  it  did  not  include  any  funds  for 
production  bomber  buys. 

However,  before  the  final 
appropriations  were  passed,  two  important 
events  took  place.  First,  East/West  tension 
increased  drastically  as  situations  in  Berlin 
worsened.  In  August,  the  Berlin  Wall  was 
constructed  through  the  city,  dividing  it 
between  East  and  West.  Fears  grew  that 
there  would  be  a  U.S./Soviet  military 
showdown. 

Second,  and  probably  more 
significantly,  was  the  Soviet  Airshow  at  the 
Tushino  Airfield.  Following  their  recent 
achievement  of  putting  the  first  man  in  space 
(Yuri  Gagarin,  April  1961)  this  airshow  was 
designed  to  show  that  Russia  was 
unquestionably  the  world  leader  in  aerospace 
technology.  The  Soviets  proudly  paraded 
their  latest  achievements  in  military  aircraft. 
Among  the  aircraft  demonstrated  was  a 
Mach  2.5  medium  bomber  larger  than  the 
B-58,  and  a  giant  supersonic  heavy  bomber 
(NATO  code-named  Blinder  and  Bounder, 
respectively).  Several  key  members  of 
Congress  became  convinced  that  the  U.S. 
must  increase  spending  for  bomber  programs 
in  order  to  stay  abreast  of  the  aircraft 
developments  demonstrated  at  Tushino. 

As  a  result  of  this  apparent  Soviet 
military  buildup  in  bomber  aircraft  and  the 
Berlin  crisis,  increased  appropriations  were 
introduced  in  Congress.  General  LeMay, 
now  Air  Force  Chief  of  Staff,  was  once 
again  called  upon  to  testify.  Regarding  the 
B-70,  he  asked  for  an  additional  $2(X) 
million  for  1962  funds,  and  $5(X)  million  for 
the  following  year.  After  the  hearings,  the 
House  Committee  stated  the  Defense 
Department  was  not  wise  in  cancelling 
production  of  the  B-58  and  B-52  bombers, 
while  at  the  same  time  restricting  the 
development  of  the  B-70.  They  also  stated 
the  first  use  of  available  funds  should  go 
towards  accelerating  the  B-70  program. 
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Meanwhile,  the  Administration  was 
also  looking  at  the  defense  budget  situation. 
On  July  31,  President  Kennedy  formally 
asked  Congress  to  increase  the  defense  funds 
for  several  programs,  but  not  for  any  bomber 
programs. 

Congress  approved  the  President’s 
requests,  and  went  one  step  farther.  The 
defense  budget  would  include  an  additional 
$180  million  for  the  B-70,  and  also  voted 
over  $500  million  for  continuation  of  the  B- 
52  and  B-58  production  lines. 

Based  upon  Congress’  insistence  on 
funding  the  B-70,  Defense  Secretary 
McNamara  undertook  another  investigation 
into  the  program.  In  January  of  1962, 
McNamara  testified  before  the  House  Armed 
Services  Committee,  stating,  "We  have  again 
restudied  the  role  of  the  B-70  in  our  future 
strategic  retaliatory  forces  and  again  have 
reached  the  conclusion  that  the  B-70  will  not 
provide  enough  of  an  increase  in  our 
offensive  capabilities  to  justify  its  very  high 
cost...."(9:373) 

Despite  the  warnings  that  he  was 
defying  the  wishes  of  Congress  and  ignoring 
the  requirements  of  the  Air  Force, 
McNamara  announced  that  he  would  again 
impound  the  additional  B-70  funds 
authorized  by  Congress,  as  well  as  the  funds 
for  further  bomber  production. 

1962 

Robert  McNamara’s  refusal  to  spend 
the  appropriated  bomber  funds  infuriated 
many  important  members  of  Congress.  It 
seemed  likely  that  some  sort  of  showdown 
would  occur  between  the  Defense 
Department  and  the  Legislature  when  the  B- 
70  program  was  brought  before  Congress  for 
the  FY  1963  budget. 

With  the  prospects  of  the  B-70 
bomber  ever  replacing  the  B-52  as  the  SAC 
mainstay  looking  bleak,  the  Air  Force 
worked  with  North  American  Aviation  to 
develop  additional  capabilities.  Added 


capabilities  and  flexibility  would  hopefully 
interest  the  Administration. 

In  1962,  the  Air  Force  proposed  the 
new  project:  a  reconnaissance-strike  version 
of  the  aircraft,  designated  the  RS-70.  The 
RS-70  was  designed  to  overfly  and  scan  the 
enemy  territory  during  or  after  a  nuclear 
exchange,  identifying  and  attacking  any 
targets  not  destroyed  by  the  first  salvo  of 
missiles.  By  processing  reconnaissance  data 
within  an  airborne  strike  vehicle,  targets 
could  be  immediately  attacked,  instead  of 
wasting  precious  time  waiting  for  the  next 
round  of  missiles.  Advanced  computerized 
radar  would  direct  airborne  missiles  to  their 
targets.  The  Air  Force  and  North  American 
were  proposing  a  program  of  60  operational 
aircraft  by  1969,  at  an  estimated  cost  of  $50 
million  each,  with  another  150  airplanes 
delivered  in  1970.  For  this  RS-70,  the  Air 
Force’s  FY  1963  request  totaled  $573.8 
million,  compared  to  only  $171  million 
allocated  according  to  the  President’s  budget. 

The  Administration  did  not  support 
the  new  and  improved  B-70.  First  of  all, 
Robert  McNamara  did  not  believe  the 
technology  required  for  such  a  mission 
existed.  He  argued  that  as  the  RS-70  flew  at 
70,000  feet  and  2,000  miles  per  hour,  the 
proposed  radar  would  be  seeing  new  areas  at 
a  rate  of  750  million  square  feet  per  second. 
The  technology  required  to  gather,  process, 
and  display  this  data  would  not  be  available 
by  the  time  the  RS-70  required  it. 

Second,  even  if  it  was  feasible,  what 
overall  contribution  would  the  RS-70  make? 
In  its  intended  mission,  the  RS-70  would  be 
hunting  for  the  last  few  surviving  and 
unlaunched  Soviet  missile  sites.  The 
importance  of  overflying  Soviet  targets 
looking  for  residual  weapons,  after  each 
country  had  already  exchanged  several 
thousand  megatons  of  nuclear  firepower  was 
compared  to  the  several  billion  dollars  of 
national  resources  required. 


51 


In  February,  General  LeMay  went  to 
Capitol  Hill  to  present  his  case  for  pursuing 
the  RS-70.  Although  the  House  and  Senate 
Armed  Services  Committees  had  agreed  with 
the  President’s  "cut-back"  B-70  program  the 
previous  year,  this  time  they  would  side  with 
the  Air  Force. 

The  House  Armed  Services 
Committee  was  chaired  by  eighty-four  year 
old  Carl  Vinson  of  Georgia.  Vinson  had 
served  in  Congress  forty-eight  years,  longer 
than  any  other  member.  Through  seniority, 
he  had  amassed  great  political  instinct  and 
power.  He  was  known  as  the  "Swamp  Fox" 
for  his  cunning  and  slyness,  and  "Admiral 
Vinson"  for  his  strong  control  of  defense 
programs.  Vinson  agreed  with  McNamara’s 
that  much  of  the  equipment  for  the  RS-70 
had  yet  to  be  developed.  In  fact,  he  used  it 
as  an  argument  to  fund  the  project.  "It  is  for 
this  very  reason  that  we  need  the  larger 
program  for  the  RS-70,"  Vinson  said  on  the 
House  floor.  He  was  concerned  that  the 
manned  bomber,  the  only  proven  strategic 
weapon,  appeared  to  be  the  most  neglected 
weapon  in  the  arsenal.  Although  he  was 
angry  at  the  President’s  decision  to  de- 
emphasize  the  manned  bomber  role,  his  real 
crusade  concerned  the  President’s 
Constitutional  authority  to  do  so. 

Vinson  had  agreed  with  the 
President’s  B-70  plan  just  the  previous  year. 
However,  he  was  now  concerned  about  the 
eroding  powers  of  the  Legislative  Branch. 
"To  any  student  of  government,"  his 
Committee  report  stated,  "it  is  eminently 
clear  that  the  role  of  the  Congress  in 
determining  national  policy  has  deteriorated 
over  the  years"  (39:19).  The  previous  year. 
Congress  had  added  over  $700  million  for 
B-70,  B-52,  and  B-58  procurement,  and 
"...not  one  penny  has  been  spent."  Using 
the  RS-70,  Vinson  now  sought  to  address 
"...a  question  more  fundamental  than  whether 
this  weapon  system  or  any  other  should  be 


adopted  as  a  part  of  our  military 
establishment"  (39:1). 

Vinson’s  House  Armed  Services 
Committee  authorized  the  President’s 
requested  $171  million,  then  voted  an 
additional  $320  million.  Furthermore,  the 
Committee’s  appropriation  "directed, 
ordered,  mandated,  and  required"  the  Air 
Force  Secretary  to  spend  the  money  and 
accelerate  the  RS-70  program.  The 
Committee  report  stated:  "If  this  language 
constitutes  a  test  as  to  whether  Congress  has 
the  power  to  so  mandate,  let  the  test  be 
made  and  let  this  important  weapons  system 
be  the  field  of  trial.  Perhaps  this  is  the  time 
to  reexamine  the  role  and  function  of 
Congress,  and  discover  whether  it  is  playing 
the  part  that  the  Founding  Fathers  ordained 
that  it  should.  "  (39:19) 

Constitutional  experts  believed  the 
Executive  Branch  would  win  Vinson’s 
challenge  if  it  went  to  the  courts.  But 
President  Kennedy  wanted  to  avoid  a  fight. 
Any  tangle  with  Vinson  and  the  Congress, 
win  or  lose,  would  be  costly  in  the  future. 

On  March  1,  Vinson’s  directive  was 
approved  unanimously  by  the  Committee, 
and  would  be  voted  on  later  by  the  entire 
House.  But  Carl  Vinson’s  challenge  angered 
some  other  members  of  Congress.  Members 
of  the  House  Appropriations  Committee 
believed  their  committee  dictated  how  funds 
were  to  be  spent.  They  felt  Vinson’s 
committee  was  exceeding  its  proper  role,  and 
doing  so  at  the  expense  of  the  Appro¬ 
priations  Committee. 

On  the  eve  of  the  House  vote,  a 
meeting  took  place  at  the  White  House. 
President  Kennedy  had  invited  Carl  Vinson 
for  a  personal  chat  in  the  Rose  Garden. 

The  President  and  Carl  Vinson  stated 
their  views  and  rationale,  and  both 
maintained  their  respective,  inflexible 
positions.  Kennedy  stated  the  country  didn’t 
need  the  RS-70,  and  as  Commander  in 
Chief,  he  would  not  .spend  the  money. 
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Vinson  responded  saying  the  countiy  did 
need  the  RS-70,  and  Congress 
Constitutionally  had  the  power  to  equip  the 
armies,  and  could  therefore  order  the 
expenditure  of  funds.  (37) 

Both  President  Kennedy  and  Vinson 
recognized  there  was  no  way  to  resolve  the 
conflict  other  than  litigation  before  the 
Supreme  Court.  Furthermore,  they  both 
believed  that  resolving  this  type  of  conflict 
between  two  equivalent  branches  of  the 
Government  was  not  the  intended  function  of 
that  court.  Vinson  agreed  they  should  not 
confront  the  court  with  this  issue,  and  agreed 
to  withdraw  the  language  "directing"  the 
spending  of  funds.  "I  ltd  those  troops  up 
that  hill,  and  by  God,  I’ll  turn  them  around 
and  lead  them  down!"  Vinson  promised  (37). 
In  exchange,  the  Defense  Department  would 
initiate  an  immediate  reinvestigation  of  the 
necessity  for  the  RS-7n/B-70.  Under  the 
agreement,  the  President  would  spend  more 
on  the  RS-70  program  if  this  new  review 
warranted  an  increase.  Furthermore,  if 
technological  developments  advanced  faster 
than  expected,  the  Administration  would 
request  and  expend  larger  funds. 

The  following  day,  the  Armed 
Services  Committee  removed  the 
controversial  language  from  the 
ammendment.  Once  the  President  had 
reaffirmed  his  power  to  withhold 
appropriated  funds,  there  remained 
practically  no  possibility  of  expanding  the 
RS-70  program. 

This  appeared  to  be  a  big  victory  for 
the  President.  The  Administration’s  promise 
to  restudy  the  program  was  essentially  "...a 
promise  of  no  value,  for  Defense  Secretary 
McNamara  had  been  studying  the  RS-70 
from  the  time  he  had  taken  office,  and  his 
only  conclusion  had  been  that  the  whole 
program  should  be  cut  back  and  probably 
killed"  (43:296).  Vinson,  on  the  other  hand, 
saw  it  as  a  victory  for  Congress.  He  told  the 
House  he  had  not  intended  to  push  the  fight 


to  a  Constitutional  showdown,  but  rather 
force  the  Pentagon  to  respect  and  respond  to 
Congressional  actions.  Meanwhile,  President 
Kennedy  stated  at  a  press  conference  that 
neither  side  had  won  or  lost.  "The  country 
was  the  winner,"  he  said,  "because  such  a 
conflict  had  been  averted"  (3:1). 

1963 

Robert  McNamara  had  appointed  Dr. 
Jo.seph  V.  Charyk,  Undersecretary  of  the  Air 
Force,  to  conduct  the  RS-70  restudy.  In 
January,  McNamara  went  before  the  House 
Armed  Services  Committee  with  the  results. 
He  opened  by  stating  the  issue  was  not  the 
future  of  manned  bombers  in  an  era  of 
missiles.  Instead,  it  was  whether  the  B- 
70/RS-70  would  add  enough  to  the  country’s 
strategic  strength  to  justify  its  high  cost. 

The  review  did  not  justify  an 
accelerated  RS-70  program,  but  McNamara 
stated  he  would  spend  $50  million  of  the 
extra  appropriations  to  further  develop 
sensor  components.  The  RS-70  remained 
programmed  for  three  test  aircraft. 

In  February,  General  LeMay  returned 
to  the  House  Armed  Services  Committee  and 
resumed  his  battle  for  the  RS-70.  LeMay 
stated  he  was  not  opposed  to  missiles; 
however,  he  felt  the  flexibility  of  a  manned 
bomber  force  was  essential.  He  asked  for  an 
expansion  of  the  program  to  five  aircraft 
instead  of  the  three  currently  programmed 
for.  The  Committee  concurred  with 
LeMay’s  arguments,  calling  the  Defense 
Department’s  increasing  emphasis  on 
missiles  over  bombers  "...a  most  dangerous 
course  of  action..."  (9:411). 

Committees  in  both  the  Hou.se  and 
Senate  approved  $364  million  above  the 
President’s  budget  for  the  RS-70  program. 
This  expansion  would  increase  the  program 
from  three  to  five  aircraft,  with  the  last  two 
upgraded  with  full  RS-70  reconnaissance 
features. 
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In  California,  the  first  XB-70  aircraft 
encountered  significant  technical  problems. 
Severe  corrosion  occurred  between  different 
grades  of  stainless  steel  and  other  materials 
in  the  airframe.  Also,  North  American 
found  difficulty  sealing  microscopic  holes  in 
the  fuel  cells.  These  problems  were 
delaying  the  plane’s  first  flight  well  into 
1964. 

Meanwhile,  the  Administration  and 
SAC  were  now  considering  concepts  for  a 
different  bomber  which  would  satisfy 
McNamara’s  criticisms  of  the  B-70.  It 
would  penetrate  Soviet  air  defenses  by  flying 
at  low  altitudes,  under  the  radar  networks. 

In  May,  Robert  McNamara 
announced  plans  to  cancel  the  RS-70 
program  outright.  In  light  of  the  new 
National  SST  project  and  assurance  that 
another  advanced  bomber  program  was  in 
work,  the  Congress  was  now  willing  to 
concede.  The  RS-70’s  main  advocate  in 
Congress  was  no  longer  able  to  rally  his 
fellow  representatives  for  his  cause.  Carl 
Vinson  had  made  a  number  of  enemies  in 
the  House.  He  had  strongly  endorsed  an 
enlarged  and  more  powerful  House  Rules 
Committee  in  1962,  a  move  that  turned 
many  of  his  House  supporters  against  him. 
Combined  with  his  earlier  controversial 
operations  of  the  Armed  Services 
Committee,  his  power  in  the  House  was 
diminishing.  (Later  that  year,  Carl  Vinson 
announced  that  after  fifty  years  in  the  U.S. 
House  of  Representatives,  he  would  not  seek 
reelection.)  Against  the  Swamp  Fox’s 
recommendations,  the  House  reversed  its 
decision  and  withdrew  $314.3  million 
allocated  for  expanding  the  RS-70  program. 

Aviation  Week  announced  on  July 
1,  1963  that  the  battle  between  McNamara 
and  Congress  over  the  B-70/RS-70  was 
finally  over,  with  McNamara  victorious 
(57:25).  "Political  background  of  the  near- 
unanimous  RS-70  vote  was  a  desire  by  many 
conservatives  in  Congress  to  get  even  with 


Chairman  Carl  Vinson....  Representative 
Vinson  could  rally  so  few  votes  that  he  went 
out  of  town  rather  than  witness  certain 
defeat...."  (57:25) 

The  RS-70  project  was  scrapped, 
and  the  B-70  concept  was  reduced  to  two 
XB-70  airplanes. 

1964 

On  May  11,  1964  the  first  XB-70 
aircraft  was  rolled  out  of  its  Palmdale, 
California  hangar.  Since  the  program  cost 
$1.5  billion  and  produced  only  two  test  air¬ 
planes,  the  rollout  was  "visible  evidence  of 
a  humiliating  defeat  for  the  bomber  men" 
(22:90).  As  Newsweek  described  the 
situation:  "The  first  public  showing  of  the 
2000  mph  B-70  would  seem  a  triumphant 
moment  for  the  Air  Force’s  big-bomber  men 
and  their  friends  in  Congress.  For  five  years 
they  doggedly  fought  first  Eisenhower  and 
then  Kennedy  Administration  efforts  to  kill 
the  giant  delta-winged  superbomber  as  an 
unneeded  frill  in  the  age  of  missilery.  Yet 
neither  Air  Force  Chief  of  Staff  Gen  Curtis 
LeMay,  HASC  Chairman  Carl  Vinson,  nor  a 
host  of  their  vociferous  B-70  backers  even 
planned  to  visit  ...  this  week  when  the  first 
B-70  was  scheduled  to  be  rolled  out.  The 
California  rollout  shaped  up  as  more  of  a 
wake  than  a  celebration."  (22:90-91) 


CONCLUSIONS 

Contrary  to  various  authors’  theories 
and  allegations,  the  B-70  program  was  not 
cancelled  because  of  Soviet  air  defenses 
(48:24  &  4:182),  or  secretly  deferring  funds 
to  the  A-11A"F-12  program  (46:22).  Nor 
was  cancellation  based  on  actual  flight 
performance.  The  B-70  fell  victim  early  in 
development  to  lack  of  political  support. 
The  government  chose  to  allocate  scarce 
dollars  to  ICBM  development  rather  than  a 
supersonic  bomber  whose  overall 
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contribution  to  strategic  mission  was 
questionable.  Except  for  $155  million 
restored  to  the  program  in  1960,  the 
Executive  Branch  never  endorsed  the 
program.  The  Congress,  a  much  stronger 
advocate,  still  was  inconsistent  from  year  to 
year  in  its  support.  The  ultimate 
confrontation  between  the  two  government 
bodies  ended  with  the  Congress  backing 
down  in  the  Rose  Garden  meeting. 

This  meeting  stands  as  a  critical 
event  in  the  history  of  defense  system 
acquisition.  Vinson’s  challenge  would  have 
settled  (either  by  judicial  resolution  or  by 
political  precedent)  the  Constitutional  law 
issue  of  Executive  discretionary  power 
versus  Legislative  mandate  over  defense 
dollars.  In  other  words,  does  the  Legislature 
require  or  permit  the  Executive  to  spend 
defense  funds?  This,  in  turn,  might  have 
begun  an  adversarial  relationship  between 
President  Kennedy’s  Administration  and  the 
Congress.  Had  Vinson  and  the  Congress 
pursued  this  challenge,  the  B-70  bomber 
program  may  have  been  expanded, 
eventually  reaching  full  production.  Instead, 
this  meeting  was  a  crushing  blow  to  hopes 
of  resurrecting  the  aircraft  as  a  weapons 
system. 

For  a  technologically  advanced 
weapon  system  to  become  part  of  the 
operational  military  inventory,  the  political 
climate  must  support  its  development  and 
procurement.  Competing  technologies  can 
bleed  funding  away,  and  eventual  benefits 
must  support  the  high  costs  likely  associated 
with  the  system’s  development  and 
production.  Without  the  consistent  support 
of  the  political  environment,  advances  in 
military  weaponry  can  not  cross  over  from 
innovative  concepts  to  operational  systems. 
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RESEARCH  CONCERNING  PROGRAM  MANAGEMENT  PRINCIPLES 


John  J.  Bennett,  DBA,  ANADAC,  Inc. 


INTRODUCTION 

General  system  theory  is  considered  a 
management  discipline  by  most  management 
specialists.  As  a  discipline,  general  systems 
theory  provides  a  body  of  knowledge,  a  set 
of  basic  concepts,  and  supporting 
management  principles.  Program  and 
project  management,  as  practiced  in 
managing  major  weapon  systems  and  similar 
developments  can  be  considered  variants  of 
general  systems  theory. 

The  (general)  system  management  practices 
in  use  by  the  Federal  Government  and  the 
aerospace/defense  industry  have  been 
evolving  since  shortly  after  World  War  II. 
The  Defense  Systems  Management  College 
(DSMC)  has  been  the  Department  of 
Defense  (DoD)  center  of  excellence  for  DoD 
program  management  education  and 
knowledge.  DSMC  first  started  training 
program  managers  in  1972.  Over  the  years 
its  training  documentation  has  undergone 
continuous  research  and  updating  to  interpret 
program  management  policy,  specify 
functional  details,  identify  practices  and 
procedures  and  provide  methods  and 
techniques.  Today,  there  are  very  few  who 
would  disagree  that  the  DSMC  body  of 
knowledge  is  an  acceptable  standard  for 
defining  program  management  as  a  unique 
discipline. 

In  review  of  overall  DSMC  documentation 
and  discussion  with  DSMC  staff,  one  finds 
that  for  the  most  part,  the  DSMC  body  of 
knowledge  is  defined,  in  general,  by  a  set  of 
functional  entities  (e.g.,  engineering, 
contracting,  etc.)  and  supporting 


management  practices.  If  one  compares  this 
with  a  notional  structure  for  defining 
management  disciplines  as  in  Figure  1,  a 
question  is  raised  as  to  how  basic  concepts 
and  management  principles  individually  and 
collectively  interact  in  the  DSMC  structure. 
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Figure  I.  Program  .Management  Knowledge  Slruclurc  Compared 
to  a  Notional  Basis  for  Defining  Disciplines 


While  acknowledging  the  value  of  the 
present  DSMC  body  of  knowledge  and  its 
structure,  a  question  can  be  raised  as  to 
whether  an  explicit  set  of  program 
management  concepts  and  principles  can  be 
defined  and  whether  these  would  contribute 
to  increasing  professionalism  and 
improvements  in  education  and  training. 
Accepting  the  general  system  or  systems 
management  concepts  as  a  foundation,  the 
research  can  be  narrowed  to  the  definition  of 
management  principles. 

THE  RESEARCH  DATA  BASE 

The  purpose  of  the  research  is  to  define  a 
set  of  unique  management  principles  that 
would  simplify  the  structural  basis  for 
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program  management  and  provide  more 
clarity  of  the  program  management 
fundamentals.  It  was  thought  that  this 
simplification  and  clarity  would  be  most 
useful  as  the  basis  for  a  training  manual  for 
staff  members  directly  related  to  program 
management  such  as  contract  specialists  and 
budget  analysts. 

The  DSMC  series  of  technical  training 
manuals  (e.g.,  the  Systems  Engineering 
Management  Guide)  and  other  substantive 
DSMC  publications  were  used  as  the  data 
base  for  this  research.  These  were 
augmented  by  a  review  of  management 
articles  from  DSMC’s  Project  Manager  and 
NCMA’s  Contract  Management  magazines 
dating  from  1982  onward. 

The  writer  attempted  to  synthesize  this  data 
around  program  management  themes  (e.g., 
designing  in  quality)  and  the  tasks  required 
to  accomplish  that  theme  (e.g.,  structuring 
program  test  and  evaluation).  The 
management  principles  postulated  and 
highlights  of  the  rationale  used  in  making 
the  selections  are  discussed  below. 

MANAGEMENT  PRINCIPLES 

Of  the  management  principles  selected 
several  are  generally  accepted  as  such  today 
(e.g.,  single  authority  and  responsibility). 
Others  are  implied  by  wide  use  (e.g.,  life 
cycle  management).  A  third  category  of 
selected  principles  are  those  that  will  need  to 
be  debated  and  probably  recast  and  refined. 
Ten  principles  are  stipulated  for  discussion. 

1.  Integration 

Integration  is  the  principle  of  bringing 
together  heterogeneous  disciplines, 
organizations,  functions,  processes, 
technologies,  data  and/or  information,  both 
vertically  (organizations,  functions,  skills. 


budgets,  etc.)  and  horizontally  (processes, 
procedures,  events,  activities,  etc.)  to 
effectively  and  efficiently  manage  a 
program.  Program  integration  is  the  basis 
for  synchronizing  the  resources  and  program 
work  to  be  performed. 

o  Rationale.  The  theme  of 
wholeness  permeates  and  drives  acquisition 
program  management.  Integration  of 
complex  and  costly  development  over  a  long 
period  of  time  is  the  premium  task  of  the 
project  manager.  The  program  manager  is 
continually  faced  with  the  integration  of 
changing  project  office  functions  and  skills, 
as  the  product  life  cycle  changes.  The 
program  manager  must  oversee  the 
integration  of  multi-purpose  processes 
pertaining  to  organizing  and  executing  the 
work,  the  integration  of  evolving  product 
technical  documentation  and  physical 
systems  and  subsystems,  and  the  synthesis 
and  integration  of  both  technic^  and 
management  information.  Management 
systems  integration  provides  the  structure 
for  program  management  and  the  program. 
Methods  and  procedures  are  available  to 
address  each  of  the  integration  tasks. 

2.  Single  Authority  and  Responsibility 

The  single  authority  principal  vests  program 
management  decision-making,  coordinating 
and  supervision  in  a  single  individual, 
limited  only  by  pre-stated  corporate 
constraints  and  the  necessary  chartering, 
organization  and  plans  to  do  this.  The 
primary  benefit  of  a  program  management 
organization  is  in  providing  a  focal  point  for 
authority,  responsibility,  coordination,  and 
ultimately  accountability  for  the  success  of 
the  program. 

o  Rationale.  Federal  Government 
policy  stipulates  single  authority  and 
responsibility  as  an  aspect  of  its  acquisition 
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management  policy.  A  decision  to  apply 
program  management  concepts  within  a 
Government  or  corporate  entity  will  in 
almost  every  case  have  a  major  impact  on 
the  corporate  organization,  and  its 
delegation  of  authority  and  responsibility. 
While  a  number  of  people  feel  recent 
pronouncements  restrict  program  managers 
to  the  exercise  of  influence  over  the 
programs  and  events,  it  can  be  pointed  out 
that  acting  as  the  program  coordinator  is  a 
central  and  an  important  task.  Program 
authority  and  responsibility  dictates 
communication  channels,  funding  and 
contracting  arrangements,  and  the  oversight 
of  engineering  and  development.  Sufficient 
documentation  and  methods  are  available  to 
address  all  of  these  considerations.  Note 
that  human  behavior  aspects  are  not 
currently  part  of  this  or  other  definitions 
herein. 

3.  Life  Cycle  Management 

The  life  cycle  management  principle 
provides  the  structure  for  managing  the 
program  process  and  sub-processes  from 
inception  of  an  idea  through  disposal  or 
replacement  of  a  product  after  its  useful  life 
expires.  Process  management  methods  are 
used  to  identify  and  resolve  functional  life 
cycle  interfaces  and  interdependencies,  as 
necessary,  to  tailor  the  product  life  cycle. 

o  Rationale.  Life  cycle 

management  implement  one  aspect  of 
program  integration  in  that  it  provides  the 
documentation  and  visible  program 
structure.  Each  common  product  (e.g., 
weapon,  space,  energy,  computer, 
information  system  or  facilities)  has  a 
common  life  cycle.  These  common  life 
cycles  have  natural  phases  they  follow  with 
only  slight  deviation.  Process  management 
methods  are  used  to  analyze  both  the 
product  and  functional  life  cycles  to  identify 


the  inter-relationship  between  the  overall 
program/functional  events  and  activities. 
Multi-functional  interfaces  and 
interdependencies  are  identified  and  related 
problems  resolved.  This  results  in  a 
program  "road  map"  which  serves  as  the 
program  master  schedule.  Further  process 
analyses  identifies  the  work  to  be 
accomplished  and  physical  processes.  The 
physical  product  technical  documentation 
results  in  a  father/son  work  breakdown 
structure  (WBS).  These  structures  -  master 
schedule,  work  to  be  accomplished  and 
WBS  -  as  a  composite,  are  part  of  and  form 
the  basis  for  life  cycle  management.  The 
composite  structure  and  data  can  be 
subjected  to  cost  analysis  to  estimate  not 
only  the  development  but  also  the 
production,  operation  and  support  costs 
(i.e.,  the  life  cycle  costs). 

4.  Pre-Specified  Objectives 

The  principle  of  pre-specified  objectives 
requires  early  on  and  specific  planning  to 
identify  explicit  technical  and  business 
requirements,  program  strategy  and  risks. 
Requirements  for  change  or  new  capability 
in  the  form  of  plans  must  be  well 
documented.  A  strategy  stating  how  the 
system  or  product  will  be  acquired  must  be 
formulated.  Program  risks  must  be 
identified  and  reduced  to  acceptable  levels 
prior  to  development. 

o  Rationale.  The  central  theme  of 
program  management  action  is  to  bring 
about  change,  and  generally  in  the  form  of 
a  new  or  modified  material  item  or  thing 
(e.g.,  a  system  or  product).  The  desired 
change  is  stated  by  setting  forth  pre¬ 
determined  objectives  for  the  item.  A 
requirement  for  a  new  or  improved  system 
or  product  can  be  generated  by  the  threat  of 
an  adversary  or  competition,  advances  in 
technology,  the  need  to  modernize  or 
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upgrade  a  system  or  product,  or  expected 
improvement  in  the  return  on  investment. 
As  the  concept  formulation  proceeds, 
alternate  concepts  are  identified  and 
explored  to  satisfy  the  pre-stated 
requirements  (i.e.,  needs).  An  acquisition 
strategy  becomes  part  of  the  planning  to 
guide  the  program  and  emphasize  transition 
into  verification  and  demonstration  of  a 
preliminary  technical  design.  Many  times 
this  is  supported  by  a  demonstration 
prototype.  A  program  baseline  (plan)  is 
defined  as  a  formal  agreement  between 
essential  program  participants  (e.g., 
program  manager,  corporate  staff,  and  the 
board  of  directors).  These  specified  critical 
parameters  of  a  program  that  must  be 
fulfilled  before  entering  development  or 
production.  Methods  and  procedures  are 
available  to  execute  each  of  these  tasks. 

5.  Design  as  Kev  Quality  Factor 

The  principle  of  designing-in  quality 
requires  a  total  perspective  of  customer 
requirements,  consideration  of  alternative 
solutions  and  trade-offs,  addressing 
manufacturing  and  logistics  factors  in 
design,  and  balancing  cost  and  performance 
with  customer  satisfaction.  The  design 
should  represent  the  best  trade-off  among 
the  competing  life  cycle  requirements, 
including  reliability  and  maintainability, 
producibility,  life  cycle  and  design-to-cost, 
and  operations  and  support  factors. 

o  Rationale.  Product  or  system 
design  is  one  part  of  the  systems  engineering 
process  and  is  a  common  technical  thread 
that  runs  through  the  program  management 
life  cycle  process.  The  product  design 
should  represent  the  best  trade-off  among 
the  competing  life  cycle  cost/performance 
requirements.  Optimization  of  the  design 
trade-off  process  is  the  initial  basis  for 
designing  in  quality  and  carries  throughout 


the  engineering  cycle.  Top  level 
specifications  are  further  refined  through 
iterative  analysis  and  scale  testing  and 
extended  at  various  levels  to  include 
analyses  of  technical  considerations  ranging 
from  human  factors  to  power  sources. 
Designing  in  reliability  and  maintainability 
(R&M)  —  the  allocation,  test  criteria  and 
planning  for  sustaining  R&M  —  are  critical 
to  quality.  As  the  design  continues  product 
availability  such  as  dimensions  of 
mechanical  structures  or  electrical 
component  tolerance  become  a  major  quality 
consideration.  Other  key  design  quality 
features  include  designing  for  production 
(i.e.,  producibility)  and  design-to-cost 
(producing  at  a  specified  unit  cost).  Special 
and  unique  industrial  design  techniques  are 
available  to  address  these  technical  factors. 

6.  System  Engineering 

The  principle  of  using  systems  engineering 
methodology,  an  iterative  technical  process, 
to  transform  the  operational  or  user  need 
into  a  technical  product  description,  to 
integrate  all  program  technical  factors,  and 
to  specify  the  total  engineering  effort. 
Technical  documentation,  configuration 
management  and  technical  reviews  and 
audits  are  all  part  of  systems  engineering. 

o  Rationale.  The  systems 
engineering  practices  allows  for  the 
development  of  a  complete  set  of  system  or 
product  technical  requirements,  documented 
initially  in  a  system  specification.  This 
document  and  lower-level  documents 
emulating  from  it  follow  a  structured 
corporate  outline  format  to  ensure 
completion  and  promote  ease  of 
understanding.  In  a  complex  program  there 
may  be  as  many  as  five  types  of 
specification,  each  with  a  specific  purpose. 
At  a  point  in  the  process  systems 
engineering  provides  for  configuration 
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management,  the  identification  of  the 
functional  and  physical  characteristics  of  an 
item  during  its  life  cycle.  As  configuration 
management  expands  it  accounts  for  and 
controls  changes  to  those  characteristics  and 
maintains  records  and  reports  for  re¬ 
procurement  of  subsystems,  components  and 
parts.  Systems  engineering  provides  a 
formal  process  of  technical  reviews  and 
audits.  These  range  from  early  and 
continuing  design  reviews,  intermediate 
functional  and  configuration  audits  and 
subsequently  a  production  readiness  review 
which  serves  as  the  base  for  a  corporate 
production  commitment.  The  basic  plan 
governing  the  system  engineering  effort  is 
the  System  Engineering  Management  Plan 
(SEMP). 

7.  Affordability  and  Supportabilitv 

The  principle  of  systematically  planning  and 
controlling  technical  factors  of  ownership  in 
terms  of  affordability  and  supportability  (and 
in  commercial  terms,  competitiveness). 
Affordability  is  the  determination  that  the 
program  life  cycle  cost  (assuming  system  or 
product  effectiveness)  is  in  consonance  with 
corporate  long-range  investment  and  product 
plans.  Supportability  is  stated  in  the  form 
of  logistics  support  parameters,  wherein 
reliability  and  maintainability  become  the 
surrogates  for  supportability  throughout  the 
development,  test,  and  production  (life 
cycle)  phases. 

o  Rationale.  The  manufacturing 
(production)  and  logistics  support  phase  of 
acquisition  program  management  are  treated 
as  part  of  the  life  cycle  management 
principle  in  this  document.  Both  functions 
are  included  as  major  elements  of  the 
systems  engineering  process.  Manufacturing 
and/or  logistics  functions  may  not  be  a 
major  part  of  another  product  life  cycle 
(e.g.,  a  one  of  a  kind  space  system  or  a 


major  software  program).  Affordability  can 
act  as  a  less  complex  surrogate  for 
manufacturing  as  can  supportability  for 
logistics.  Both  are  key  cost  drivers  for  the 
program.  Affordability  and  supportability 
involve  considerations  and  decisions  at  the 
decision-maker,  project  manager  and 
contractor  levels  (albeit  somewhat  different). 
Both  can  highlight  performance  (e.g.,  kill 
effectiveness  per  dollar),  are  amenable  to 
mathematical  formulation,  and  can  be 
measured.  As  concepts,  both  are  easy  to 
remember  (but  somewhat  more  difficult  to 
assimilate).  Both  can  be  considered  element 
of  commercial  competition.  They  address 
two  basic  questions;  can  the  customer 
afford  my  product  as  compared  to  my 
competitors,  and  can  it  be  operated  and 
maintained  effectively  (supported)  over  its 
usage  period? 

8.  Structured  Test  and  Evaluation 

The  principal  of  structured  program  life 
cycle  test  and  evaluation  provides  the  major 
method  of  program  risk  management. 
Costly  product  re-design  or  modification  can 
be  reduced  if  properly  planned.  As  a  risk 
identification  and  reduction  tool,  test  and 
evaluation  (T«S:E)  provides  information  to 
decision-makers  responsible  for  deciding  on 
the  most  effective  use  of  resources. 

o  Rationale.  Risk  management  is 
the  means  by  which  the  program  areas  of 
product  performance  and  uncertainty  are 
identified  and  managed.  T&E  is  the 
discipline  that  helps  illuminate  these  areas  of 
vulnerability.  While  the  terms  "test"  and 
"evaluation"  are  most  often  found  together, 
they  actually  denote  clearly  distinguishable 
functions.  "Tests"  denote  the  actual  test  of 
hardware/ software  -  models,  prototypes, 
production  equipment,  computer  programs 
to  obtain  technical  data,  etc.  "Evaluation" 
denotes  the  review  and  analysis  of 
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qualitative  and  quantitative  data  obtained 
from  design  reviews,  hardware  inspection 
and  testing  and  operational  usage  of 
equipment.  Today,  T&E  has  reached  a 
stage  of  importance,  in  meeting  pre-specified 
objectives  requiring  separate,  detailed  test 
and  evaluation  life  cycle  plans.  Within  the 
Federal  Government,  the  Congressional 
requirement  for  T&E  reporting  from  the 
DoD,  and  DoD’s  establishment  of 
independent  T&E  organizations  within  its 
components,  reflect  decision-makers  interest 
in  assessment  and  management  of  systems 
utility  and  effectiveness.  While  the 
structured  life  cycle  T&E  program  provides 
the  major  sources  of  formal  risk 
management,  aspects  of  risk  management 
are  inherent  also  in  other  program 
management  functions  (e.g.,  identification 
and  analysis  of  programmatic  risks  by 
budget  analysts). 

9.  Program  Control 

The  principle  of  program  control  involves 
the  use  of  management  techniques  and 
information  for  planning  and  controlling 
program  schedules,  cost  and  technical 
performance  (and  related  risks).  Program 
control  includes  cost  estimating,  milestone 
and  master  plan  scheduling,  PERT/CPM, 
and  cost  and  schedule  control  techniques 
(and  related  technical  status). 

o  Rationale.  Program  managers  are 
responsible  for  technical  management, 
business  management  and  program 
advocacy.  Program  control  is  the  major 
business  management  function  and  centers 
on  the  cost  and  schedule  control  procedure. 
Other  business  management  functions 
include  financing,  contracting, 
administration,  and  data  management.  Most 
of  the  latter  functions  are  corporate  tasks 
with  internal  input  and  assistance  from  the 
program  office.  Complexity  and  uncertainty 


in  most  major  programs  can  lead  to 
continually  increasing  product  costs,  a  key 
concern  for  program  management.  Program 
managers  must  have  an  effective  internal 
cost  and  schedule  management  control  (i.e., 
program  control)  system  for  all  aspects  of 
work.  In  many  cases,  program  control  must 
rely  on  timely  and  auditable  input  data  from 
its  contractor  systems.  A  formal  program 
control  procedure  includes  use  of  program 
work  breakdown  structures;  the  concept  of 
work  packages  at  the  lowest  level  of 
management;  work  measurement,  earned 
value  and  estimate-to-complete  techniques; 
a  relationship  to  technical  status  and 
technical  performance  measurement;  and,  an 
integrated  audit  and  reporting  system. 
Control  of  schedules,  cost  and  technical 
performance  is  one,  if  not  the  most,  critical 
tasks  of  program  managers. 

10.  Continuous  Management 
Improvement 

This  principle  reflects  the  need  for  program 
managers,  collectively  and  individually,  to 
be  informed  about,  to  promote  and  to  use 
new  automation  technology  to  improve  the 
effectiveness  and  efficiency  of  program 
management  and  the  physical  processes  used 
for  developing,  producing  and 
distributing/deploying  the  system  and 
products  they  are  managing.  Computed 
Aided  Design  (CAD),  Computer  Aided 
Manufacturing  (CAM)  and  Computer 
Acquisition  logistics  Systems  (CALS)  are 
three  of  the  major  new  automation  initiatives 
being  developed  and  used  in  program 
management. 

o  Rationale.  Management  in 
general  has  undergone  a  revolution  in  its 
content  and  the  application  of  new 
management  techniques.  Computer, 
communication  and  information  technology 
is  affecting  industrial  organizations, 
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processes  and  methods  as  digital  automation 
applications  evolve.  No  where  is  this  more 
evident  and  critical  then  in  major  program 
management  activities.  Both  physical 
engineering  and  business/technical 
information  process  improvements  are 
critical  to  improving  productivity  and 
reducing  program  complexity  and  costs. 
Program  management  is  expecting  to  enter 
an  age  of  near-paperless  processes  over 
time.  Computer-aided  design  (and 
manufacturing)  will  provide  technical 
documents  and  configuration  data  in  digital 
form.  Computer-integrated  manufacturing 
technology  will  provide  for  automated 
factories.  In  turn,  the  computer  acquisition 
logistic  system  will  improve  almost  all 
aspects  of  follow-on  material  management. 
Program  managers  leadership  in 
understanding,  applying  and  nurturing  these 
developments  can  be  critical  to  progress  and 
have  a  composite  impact  on  U.S. 
competitiveness. 

SUMMARY 

The  writer  states  that  the  program 
management  discipline  as  defined  by  DSMC 
documentation  is  based  on  a  composite  of 
related  functional  disciplines  and  supporting 
management  practices.  He  hypothesizes  that 
a  set  of  program  management  principles  can 
be  developed  from  the  DSMC  body  of 
knowledge  that  would  simplify  the  basic 
structure  and  clarify  the  understanding  of  the 
program  management  fundamentals.  In  his 
research  and  analysis  he  postulates  a  set  of 
ten  principles  and  the  rationale  for  the 
selection  of  each.  He  draws  no  conclusions. 

The  writer  hypothesizes  several  possible 
values  in  using  a  set  of  management 
principles  to  link  the  program  management 
body  of  knowledge  and  its  supporting 
structure.  If  a  .set  of  management  principles 
can  be  defined  and  agreed  on,  they  r  ay 


provide  a  clearer  line  of  demarkation 
between  program  management  and  other 
management  disciplines  and  functional 
entities.  Having  narrowed  the  art  of 
program  management  to  a  limited  set  of 
descriptors,  it  would  seem  program 
management  could  claim  and  justify  a  high 
degree  of  uniqueness.  As  each  program 
management  principle  is  extended  to 
successive  lower  levels  of  detail,  less 
overlap  with  other  functions  should  occur. 
Finally,  education  and  training  learning 
objectives  under  this  approach  could 
probably  be  stated  more  clearly  and  more 
easily  measured  during  performance. 

It  is  recommended  that  DSMC,  NCMA  and 
other  interested  parties  employ  a  volunteer 
group  of  program  management  experts  to 
investigate  the  propositions  and  hypotheses 
as  set  forth  herein. 
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ACTIVITY  BASED  COST  ACCOUNTING 


Norman  E.  Bielmeier 
Chie-f,  Contract  Operations  Branch 
DCMO  But talc 
DCMAO  Syracuse 
DCMD  Northeast 


ABSTRACT 


With  the  advent  of  Computer 
Integrated  Management  (CIM)  Systems, 
Robotics  and  flexible  manufacturing 
lines,  less  direct  factory  labor  is 
necessary.  Many  Defense  Contractors 
have  automated  their  factories; 
others  are  in  the  process  of 
automation  and  there  are  those  still 
investigating  the  various 
possibi 1 ities. 

It  has  became  evident  to  most, 
how^^ver,  that  the  use  of  traditional 
cost  accounting  systems  in  a  highly 
automated  environment  can  lead  to 
cost  distortion  when  a  traditional 
labor  base  system  is  used  to 
accumulate  costs. 

This  article  explains  Activity  Based 
Costing  (ABC) ,  where  it  came  from 
and,  what  it  can  do  for  American 
Business.  Further,  it  gives  some 
examples  of  how  ABC  can  help  true 
product  costing  and  aid  management 
in  making  intelligent  business 
decisions.  Finally,  it  will  show  the 
relevance  of  Activity  Based  Cost 
Accounting  to  Defense  Contracting. 


ART I CLE 


Activity  Based  Costing  (ABC) 
is  an  accounting  procedure  that 
recognises  activities  and  processes 
as  a  more  relevant  basis  for  the 
spreading  of  overhead  than  a  direct 
labor  base  system.  The  ABC  system 


causes  the  activities  that  use 
overhead  to  be  charged  with  the 
proper  amount  of  overhead.  Kevin 
Kelly,  a  staff  writer  for  Business 
Week,  tells  us  that  the  reason  the 
Direct  Labor  Base  (DLB)  is  no  longer 
a  good  basis  for  allocation  of 
overhead  is  that  direct  labor  new 
only  15'/.  of  total  cost  in  most 
industries.  In  some  hi-tech 
industries  it  may  be  as  low  as  5’i 
(43).  Norm  Raffish,  one  of  the 
Senior  Managers  at  Ernst  and  Young, 
states  that  5'/.  to  15'/.  is  in  the 
ballpark:  (36).  Some  of  the  examples 
in  Exhibit  A  show  that  one  of  the 
local  companies  average  2’/.-3’/.  and  in 
one  case  with  setups  removed,  direct 
labor  is  only  1.3'/.  of  total  cost. 
Theoretically,  in  a  company 
utilising  a  Computer  Integrated 
Manufacturing  (CIM)  system,  the  time 
for  setups  should  be  close  to  zero 
since  after  the  machines  are 
programmed  the  first  time,  there 
should  be  virtually  no  set  up  time. 

In  automated  facto'^y 
environments  that  have  15'/.  or  less 
of  direct  labor  to  total  cost,  a  DLB 
is  inappropriate  as  means  of 
spreading  overhead;  Overhead  may  be 
as  high  as  50'/.  of  total  product 
cost.  Exhibit  B  shows  how  a  direct 
labor  based  system  arrives  at  a 
total  cost  of  $668.33  for  a  widget 
and  $1181.66  for  a  gadget.  The  total 
amount  of  overhead  to  be  spread  is 
$770.00.  The  overhead  is  then 
divided  by  the  total  direct  labor 
dollars  ($240.00)  and  that 
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■factor  is  multiplied  by  the 
individual  direct  labor  dollars  for 
widgets  and  gadgets. 

In  Exhibit  C,  a  Processing 
Time  Based  system  is  used. 
Processing  time  could  be  machine 
hours.  It  may  be  easier  to  think  of 
machine  hours  as  a  cost  driver  in  a 
factory  that  has  only  a  2'/.-3'/.  direct 
labor  base.  In  this  example,  the 
total  overhead  is  still  ^770. 00; 
however,  this  overhead  is  spread  by 
the  available  12  processing  hours. 
Dividing  the  #770. 00  bv  those  12 
hours  yields  a  factor  of  64.166. 
This  factor  is  then  multiplied  by 
the  processing  dollars  for  widgets 
and  gadgets.  You  should  note  that, 
under  the  processing  time  basis,  the 
cost  allocation  is  exactly  the 
opposite  of  the  direct  labor  base; 
the  total  cost  of  the  widget  or 
gadget  is  either  $668.33  or  $1181.66 
depending  upon  the  base  used. 
Further  note  that  the  numbers  used 
in  Kevin  Kelly's  article  in  Business 
Week  are  incorrect.  Kelly  spreads 
the  $240  dollars  of  processing  cost 
that  are  already  included  in  the 
product  cost  (43).  (Please  see 
Exhibits  B  and  C  for  an  example) 

The  ramifications  of  different 
overhead  bases  are  very  easy  to 
deduce.  Using  a  Direct  Labor  Base  to 
spread  overhead  in  a  highly 
automated  factory  will  spread  the 
overhead  incorrectly.  This  means 
that  product  pricing  will  have  no 
direct  correlation  to  the  resources 
used,  nor  will  the  price  reflect  the 
real  cost.  Some  products  will  be 
undercosted  and  some  will  be 
overcosted.  Clearly,  if  this  cost 
data  is  used  for  management 
decisions,  wrong  decisions  could  be 
made.  If  management  bases  their 
decision  on  this  cost  data  to  shut 
down  a  product  line  or  a  plant,  they 
will  make  a  serious  mistake  with  far 
reaching  effects. 


Where  did  ABC  come  from? 
Gerald  Aiyathurai,  of  Peat  Marwick, 
states  that  "Activity  Accounting" 
was  first  developed  and  introduced 
by  Eric  Kohler  in  the  1930s  (61), 
while  he  was  working  for  the 
Tennessee  Valley  Authority.  The 
concept,  devised  50  years  earlier, 
was  not  used  until  the  1980s.  Don 
Bohl  states  that  it  was  Dr.  Robert 
S.  Kaplan,  of  Harvard  University, 
who  first  sounded  the  alarm  in  the 
Harvard  Business  Review  in  August  of 
1984  (1).  Kaplan  argued  that  "cost 
distortions  produced  by  the  archaic 
accounting  system  were  hobbling 
American  competitiveness"  (Bohl  1). 
The  next  major  breakthrough  came  in 
1986  when  Computer-Aided 
Manufacturing  International  (CAM-I) 
was  formed.  CAM-I  was  initiated  as 
an  association  of  40  major 
manufacturing  and  accounting  firms. 
This  group  came  up  with  a  conceptual 
frame  work  and  better  accounting 
systems. 

Around  that  same  time.  Peat 
Marwick  came  up  with  an  ABC  service 
that  relied  on  software  developed  by 
Robert  Kaplan  and  Robin  Cooper  of 
Harvard  (Bohl  2).  This  service  finds 
the  real  cost  drivers  and  then 
implements  the  use  of  that  driver  to 
show  the  true  cost  of  goods 
manufactured. 

What  can  Activity  Based 
Costing  do  for  business?  ABC  is  the 
only  one  of  the  methods  called 
Advanced  Cost  Management  Systems 
(ACMS)  by  the  Defense  Contract  Audit 
Agency  (DCAA).  These  systems  point 
out  direct  relationships  between 
cost  accumulation  and  the  driver 
that  causes  the  cost  to  occur.  The 
essence  of  ABC  is  that  a  simplistic 
direct  causal  relationship  exists 
between  the  cost  driver  and  the 
items  being  produced.  James  A. 
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MDririt:t-a  f|ntr@»  th#t  raitiafe 

should  be  used  •for  di-fferent 
department  or  different 

manufacturing  cells  if  the  level  of 
automation  differs,  or  there  is  a 
difference  in  activities  or  cost 
drivers  (5) . 

ABC  can  show  management  how 
their  resources  are  being  used.  Once 
direct  labor  is  replaced,  as  the 
base,  with  machine  hours;  a  new 
relationship  between  the  resource 
(the  Machine)  and  the  products  is 
clearlv  seen.  If  the  machine  is  run 
on  electricit-/,  it  is  obvious  that 
the  monthly  utility  bill  is  not  a 
fixed  cost  at  all.  If  the  machine 
must  be  financed,  one  can  now  draw 
an  exact  relationship  between  the 
monthly  finance  cost  of  the  machine 
and  the  return  on  the  inyestment.  In 
other  words,  ABC  offers  a  host  of 
new,  vital  information  to  assist 
management  in  mai-ing  intelligent, 
worthwhile  business  decisions. 

Beyond  that,  from  an 
accountincj  tfieory  perspective,  true 
product  pricing  is  achieved  through 
the  use  of  ABC.  If  a  company  is  to 
become  more  competitive,  it  must 
trow  what  it's  products  really  cost. 
Robin  cooper,  of  Harvard  University, 
States  that  "Managers  in  companies 
■s.e.l,'’g  multiple  prod'ucts  are  mating 
.  tarit  decisions  about  pricing, 
P'-oducl  mi:.,  and  process  technology 
baaed  on  distorted  cost  information" 
Cooper,  19R0  (38).  Haedicke,  of 

Hjche:.  Aircra-(t.  states  that  the 
direct  1  abor  bc?sed  system  had  its 
Gener.-.l  Motors  during  the 
e*"  -  ri  51.180  and  DuPont  (30).  Recent 
;e:.  1^,0'is  Dv  General  Motors  to  sell 
i'-c  c.poe  plants  were  lilely  made 
base:;  _,n  distorted  accounting  data. 
General  is  doing  poorly  and 

.s  'ct  wit'. but  merit  to  question 
trai-  accc.int;  nq  s.stem-c.n  overhaul 
•  fat  ->/stem  could  have  made  a 
ci*^e-'ence  f.  wr.ich  plants  General 
‘•'bt;-'---  -p  close. 

all  o-t  this  relevant  to 
Ce*e'se  c'ntractirq  , n  the  future' 


MQ»t  of  our  d»fenss  contractors  use 
a  direct  labor  base  for  product 
costing.  As  contractors  continue  to 
automate,  their  direct  labor  input 
base  will  drop.  If  our  contractors 
start  using  Computer  Integrated 
Manufacturing,  their  setup  labor 
will  be  drastically  reduced,  if  not 
all  but  eliminated.  The  result  of 
this  drop  of  direct  labor  input  cost 
to  less  than  15’/.  of  total  product 
cost  will  be  a  distortion  of 
oyerhead  cost  allocation. 

What  does  this  do  to  the 
defense  industry?  Cost  data  will 
certainly  be  distorted.  Some 
products  will  be  underpriced  while 
others  will  be  oyerpriced.  Defense 
managers,  when  making  strategic 
decisions  based  on  distorted  cost 
information  will,  in  most  cases, 
make  the  wrong  decisions. 
Specialized  defense  products  taking 
a  large  portion  of  a  company's 
resources  are  yery  likely  neither 
fair  nor  reasonably  priced.  In  the 
interest  of  fair  and  reasonable 
prices,  defense  contractors  that 
highly  automate  should  investigate 
and  evaluate  ABC  costing.  Defense 
contractors  who  have  highly 
automated  production  facilities 
could  be  under  costing  their 
specialized  military  product  lines 
and  over  costing  their  non-complex 
commercial  product  lines. 
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DIHECT  LABOR  AMALYSIS 


CATEOOST 

HOUBS/ 

DMIT/ 

EXTENDED 

UHIT 

EFFCY 

COST 

MFG 

HHS 

i 

$ 

SETUP 

22.060 

0.34 

596.51 

MACHINE  SHOP 

0.054 

0.33 

578.75 

ASSEMBLY  &  TEST 

0.001 

0.01 

22.90 

23.33J  Total 

PLATE 

0.022 

0.23 

412.49 

-22.960  SetuD 

PAINT 

0.000 

0.00 

0.00 

.371 

PACKAGING 

0.052 

0.68 

1203.59 

BOBBER  MOLD 

0.002 

0.03 

53.21 

INSPECTION 

0.022 

0.39 

685.70 

23.331 

OTHEB 

0.217 

2.41 

4286.18 

1.5% 

TOTAL  MFG  LABOR 

23.331 

4.42 

7839.33 

OVERHEAD  • 

357. 9X 

15.81 

28056.98 

MFG 

HRS 

« 

> 

SETUP 

0.300 

0.01 

23.97 

ASSEMBLY 

0.501 

3.71 

6589.05 

INSPECTION 

0.007 

0.10 

182.16 

TOTAL  MFG  LABOR 

0.808 

3.83 

6795.18 

OVERHEAD  • 

400. 4X 

15.33 

27207.91 

MFG 

HRS 

• 

* 

SETUP 

0.000 

0.00 

0.00 

ASSEMBLY 

0.077 

0.60 

1067.57 

INSPECTION 

0.000 

0.00 

0.00 

TOTAL  MFG  LABOR 

0.077 

0.60 

1067.57 

OVERHEAD  • 

268. OX 

1.62 

2870.70 

TOTAL  MFG; 

t 

A 

TOTAL  MFG  LABOR: 

8.85 

15702.00 

OVERHEAD 

32.75 

58135.50 
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EXHIBIT  A 


COST  BREAKDOmS 


PART  VnifBER 

0T2146 

F282- 1120-4 

0T2160 

0T2128 

Material 

569.03 

431.83 

5857.88 

1530.40 

Labor 

16.28 

12.53 

47.06 

22.66 

Mfg.  Overhead 

44.64 

34.36 

120.04 

73.10 

Manufacturing  Cost 

620.05 

478.72 

6033.08 

1630.16 

a  &  A 

138.50 

105.32 

1327.48 

358.64 

Total  Cost 

768.54 

584.04 

7361.46 

1088.80 

Fee 

153.71 

116.81 

1472.20 

397.76 

Cost  of  Money 

1.70 

1.30 

0.21 

3.30 

Unit  Price 

023.05 

702.15 

8842.06 

2380.05 

DL/TC 

2. IX 

2.14X 

0.63X 

1.34X 

16.28 
768754 
=.021183 
OR  2.iy. 


EXHIBIT  A 
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TOTAL  COST 

DIRECT  LABOR  BASE 
WIDGETS  GADGETS 


DIRECT  LABOR 

DOLLARS 

$40 

$200 

PRODUCTION 

COSTS 

$540 

$540 

OVERHEAD 

$168 

$842 

$708 

$1382 

1010/240  =  4.20833 

N.  BIELMEIER 
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EXHIBIT  B 


ACTIVITY  BASED  COSTING 

PRODUCTION  COSTS 

WIDGETS  GADGETS 
LABOR  @  $10  /  HOUR 
$40  $200 

MATERIALS  AND  PARTS 
$300  $300 

PROCESSING  (9>  $20  /  HOUR 

$200  $40 

•HI  mm  wmt  mh  mm  mm  t^m  hm  hb  mm  mm  m^m  mmm  ^m 

$540  $540 

N.  BIELMEIER 
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EXHIBIT  B 


TOTAL  COST 

PROCESSING  TIME  BASE 


WIDGETS  GADGETS 


PROCESSING  DOLLARS 
$200  $40 

PROCESSING  TIME 
10  HRS  2  HRS 

PRODUCTION  COSTS 
$540  $540 

OVERHEAD 

$842  $168 


$1382  $708 

1010/12  HRS  =  84.2 

N.  BIELMEIER 

EXHIBIT  C 
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A  NEW  BEGINNING:  ACQUISITION  PLANNING  WITH  SOFTWARE  REUSE 
Robert  J.  Bowes,  Theresa  R.  Huber,  DSD  Laboratories,  Inc. 


ABSTRACT 

Integrating  software  reuse  principles  into 
systems  acquisition  practices  and  software 
engineering  processes  provides  a  basis  for 
dramatic  improvement  in  the  way  software¬ 
intensive  systems  are  developed  and 
maintained  over  their  life  cycle*.  The  DoD 
is  currently  developing  plans  to  incorporate 
software  reuse  into  all  classes  of 
software-intensive  systems. 

The  DoD  plans  to  require  business  managers 
(Program  Executive  Officers  (PEOs))  to 
manage  reuse  across  their  systems*.  This 
paper  discusses  how  reuse  can  be  practiced 
under  the  current  acquisition  structure  and 
proposes  a  new  structure,  which  follows 
from  the  DoD’s  central  management  theme, 
which  will  allow  software  reuse  to  be 
incorporated  more  effectively  and 
efficiently.  The  concepts  discussed  here 
were  developed  under  the  Central  Archive 
for  Reusable  Defense  Software  (CARDS) 
program,  sponsored  by  ESC/AVS,  Hanscom 
AFB,  MA.  These  ideas  are  contained  in 
two  acquisition-related  handbooks: 
Acquisition  and  Direction  Level^ 
Handbooks^. 

INTRODUCTION 

Software  reuse  is  a  process  in  which 
software  resources  are  employed  in  more 
than  one  system.  Reuse  can  occur  within  a 
system  (e.g.,  F-14A,  B,  ...),  across  similar 
systems  (e.g..  Ml,  Bradley,  ...)  or  in 
widely  different  systems*. 

Application  areas  within  which  software 
components  are  reused  are  called  domains. 


A  domain  is  a  group  of  related  systems  that 
share  a  set  of  common  capabilities.  These 
domains  can  be  defined  as  vertical  or 
horizontal  domains.  A  vertical  domain  is  a 
specific  class  of  system,  such  as  information 
systems,  command  and  control  or  weapon 
systems.  A  horizontal  domain  consists  of 
general  software  functions  that  are 
applicable  across  vertical  domains*.  Figure 
1,  Vertical  and  Horizontal  Domains, 
illustrates  the  relationship  between  vertical 
and  horizontal  domains'*. 

The  DoD  is  planning  to  take  software  reuse 
one  step  further  to  reach  systematic  reuse, 
where  opportunities  are  pre-defined  and  a 
process  for  applying  those  opportunities  is 
fully  specified*.  To  reach  systematic  reuse 
and  the  long-term  benefits  of  shorter 
schedules,  decreased  costs  and  higher 
quality  and  reliability,  the  DoD  has 
developed  a  Software  Reuse  Vision  and 
Strategy.  The  services  are  currently 
developing  compatible  supporting  reuse 
strategies  and  planning  documents. 

The  DoD  Software  Reuse  Vision  and 
Strategy  document  focusses  on  incorporating 
software  reuse  into  all  classes  of 
software-intensive  systems:  information 
systems,  command  and  control  systems,  and 
weapon  systems.  The  plan  includes  an 
analysis  of  the  business,  technical,  and 
development  context  in  which  to  implement 
a  software  reuse  strategy,  so  that  the  DoD 
can  move  towards  a  process-driven, 
domain-specific,  reuse-based,  technology- 
supported  model  for  software-intensive 
development.  To  support  software  reuse, 
the  DoD  will  invest  in  an  infrastructure  that 
centers  on  advancing  technologies  that 
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Information  Systems 


Command  and  Control 


Weapon  Systems 


Figure  1 

Vertical  and  Horizontal  Domains 


support  reuse,  incorporating  reuse  into 
management  and  engineering  processes,  and 
creating  generic  sets  of  components  to  reuse 
in  new  systems  and  in  software 
maintenance. 

The  DoD’s  vision  includes  requiring 
business  managers  to  establish  plans  to 
manage  reuse  across  their  systems'.  We 
recommend  that  a  "Domain  Management 
Office"  be  established  to  manage  the 
domain.  The  Domain  Management  Office 
would  perform  the  domain  analysis  and 
maintain  the  knowledge  and  associated 
requirements  for  a  particular  domain,  as 
well  as  work  together  with  Using 
Organizations  and  the  Program  Offices 
within  the  domain.  This  concept  as  well  as 
roles  and  responsibilities  of  executives  in 
this  proposed  structure  is  discussed  in  the 
following  paragraphs  as  a  "New  Beginning" 
to  fu  'ire  acquisitions. 

A  NEW  BEGINNING 
Integrating  software  reuse  principles  into 


systems  acquisition  practices  and  software 
engineering  processes  provides  a  basis  for 
dramatic  improvement  in  the  way  software¬ 
intensive  systems  are  developed  and 
maintained  over  their  life  cycle'. 

Implementation  of  software  reuse  at  the 
beginning  of  the  life  cycle  has  demonstrated 
a  number  of  benefits.  The  quality  and 
reliability  of  software-intensive  systems  can 
be  improved  by  using  already  proven 
software  components  or  through 
prototyping.  Existing  components  can  be 
used  to  prototype  a  system/user’s 
requirements;  thereby,  technical  risks  can  be 
identified  early  and  managed  properly  in 
order  to  reduce  negative  implications. 
System  interoperability  can  also  be  enhanced 
through  reuse.  For  example,  systems  will 
be  able  to  interoperate  more  effectively  with 
other  systems  by  using  a  single 
communications  package'.  Reusing 
pre-tested  software  components  can 
accelerate  system  development  and 
deployment  by  reducing  time  spent  in 
development,  testing,  and  maintenance. 
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resulting  in  improved  productivity.  As  a 
result,  software  acquisition  costs 
(development  and  overall  life  cycle)  are 
reduced  significantly.  New  requirements 
and  changing  technologies  can  therefore  be 
integrated  on  a  continuing  basis. 

For  reuse  to  be  effective  and  achieve  its 
broadest  influence,  Government  executives 
must  consider  it  in  the  earliest  stages  of  the 
Acquisition  Life  Cycle.  In  the  context  of 
DoD  Directive  5000. 1 ,  we  are  talking  about 
reuse  consideration  at  Milestone  0  and  at 
every  subsequent  milestone  throughout 
production  and  onto  subsequent  Post- 
Deployment  Software  Support  (PDSS).  This 
is  critical,  since  studies  today  suggest  that 
support  costs  can  reach  as  much  as  70-80% 
of  overall  life  cycle  software  costs.  To  the 
extent  that  Government  executives  can  avoid 
software  development,  improve  its  reliability 
and  create  multiple  reuse  environments 
across  (and  within)  programs,  the 
Government  will  achieve  savings  in  time  and 
dollars.  These  savings  would  be  seen 
primarily  in  systems  deployment  and 
maintenance.  In  addition,  operational 
effectiveness  would  be  increased  because  of 
improved  reliability,  standardization  and 
training. 

Conceptually,  Figure  2,  Applying  Reuse  to 
the  Current  Acquisition  Structure,  describes 
how  key  players  relate  in  the  acquisition 
structure  which  exists  today.  It  also  shows 
how  reuse  could  be  best  recognized  by 
participants  as  a  potential  solution,  and  how 
it  could  be  integrated  into  the  acquisition 
and  business  strategy  process.  Those  items 
shown  in  italics  are  not  presently  formalized 
for  reuse  in  the  current  acquisition  structure. 
What  is  most  characteristic  about  this 
structure  is  its  "ad  hoc"  nature.  Until 
recently,  there  has  been  no  systematic 
attempt  to  identify  and  integrate  reuse  as  a 
sound  systems/software  engineering  solution 


in  the  requirements  validation  and  analysis 
process.  As  a  result,  central  plamning, 
programming  and  budgeting  for  reuse  has 
been  non-existent.  A  PEO-level  program 
may  embrace  reuse  (as  is  the  case  in  the 
Army’s  Tactical  Command  and  Control 
System  (ATCCS)  and  its  five  nodal 
segments)  Success  Stories),  but  this  has 
typically  happened  because  of  the  drive  of 
individuals,  not  policy.  More  often  than 
not,  when  reuse  is  considered,  it  is  at  the 
single  program  level  by  an  individual 
program  manager.  DoD’s  new  Software 
Reuse  Vision  and  Strategy  document  will 
begin  to  change  this  policy  toward  one  that 
integrates  advocacy  at  high  levels  with  a 
proactive  approach  toward  reuse.  But,  for 
now,  reuse  will  only  be  effective  if  senior 
Government  executives  appreciate  the 
importance  of  reuse,  recognize  its  benefits, 
and  foster  its  use. 

Figure  2  today  requires  initiative  by  a  TEO 
or  Designated  Acquisition  Commander 
(DAC)  or  Program  Manager  to  assess  the 
viability  of  reuse  as  a  technical  solution  to 
user  requirements.  It  is  best  accomplished 
at  the  PEO/DAC  level,  and  later  validated 
through  analysis  and  implementation  by  the 
Program  Manager,  since  a  PEO/DAC  is 
likely  to  have  a  broader  perspective  in 
assessing  program  impacts 
(technical/schedule/cost)  of  reuse 
implementation  than  an  individual  Program 
Manager.  In  other  words,  the  PEO/DAC  is 
in  the  best  position  to  control  planned  reuse 
across  programs.  This  executive-level 
assessment  of  reuse  viability  has  the 
advantages  of:  1)  enabling  more  objective 
assessment  of  reuse  investment  value  on  a 
single  program  versus  potential  savings  in 
multiple  programs;  2)  considering 
applications  within  and  outside  the 
PEO/DAC’s  sphere  of  responsibility;  and, 
3)  assessing  the  overall  effectiveness  cf  an 
implemented  reuse  strategy. 
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Figure  2 

Applying  Reuse  to  the  Current 
Acquisition  Structure 


The  shortcomings  in  reuse 
institutionalization  today  can  be  directly 
traced  to  an  unorganized  approach  to  its 
implementation.  The  lack  of  organization 
results  from  the  ambiguity  in  responsibility 
for  reusable  components.  To  the  extent  that 
reuse  is  practiced  today,  applications  are 
program  specific.  The  Government 
Program  Manager  initiates  reuse,  the 
program  is  completed,  the  reusable 
component  is  delivered  to  a  user,  and  (with 
luck)  a  library.  The  component  is  never 
updated  (or  at  least  not  regularly).  No  one 
is  assessing  its  viability  for  similar  domain 
applications,  unless  a  specific  program 
becomes  aware  of  the  component.  The 
result  -  probably  a  one  time  or  two  time 
application,  then  the  component  is 
forgotten/abandoned. 

Some  applications  transcend  the  majority. 
The  Army  ATCCS  program  has  focused  on 
component  reuse  in  its  nodes.  The  Navy 
AX  Weapon  System  program  is  requiring 
examination  of  Air  Force  ATF  components 
for  reuse  potential.  These  are  significant 
steps  within  and  across  program  domains. 
Unfortunately,  there  is  no  clear  indication 
that  anything  will  follow  beyond  these 
applications,  since  no  comprehensive  plan  to 
institute  reuse  has  been  implemented. 

The  introduction  and  institutionalization  of 
the  Domain  Management  Office  -  an 
organization  charged  with  development, 
improvement,  ownership  and  application  of 
reusable  components  within  and  across 
programs  is  proposed.  The  previous  model 
is  repeated  here,  adding  the  concept  of 
domain  management. 

The  Reuse  Acquisition  Planning  Structure, 
Figure  3,  describes  an  environment  where 
an  organization  interfaces  with  Using 
Organizations,  PEOs/DACs,  Program 
Offices  and  industry  to: 


•  Identify  needs 

•  Perform  domain  analysis  to 
support  systems  engineering  and 
requirements  analysis 

•  Develop/identify/maintain/ 
research/upgrade  components  for 
reuse  application  within  the  domain 

•  Improve  reuse  through 
development  of  metrics,  standards, 
methodologies,  training  and  tools 
which  are  further  refined  through 
feedback 

•  Assess  the  effectiveness  of  reuse 

•  Populate  and  support  a  domain- 
specific  library  for  components 

•  Propose,  develop  and  manage  a 
budget  to  support  reuse  resources 

•  Identify  steps  to  resolve  legal 
ownership  issues  to  balance  industry 
and  Government  needs 

The  domain  management  function  would 
centrally  manage  software  reuse  for  a 
particular  domain.  Software  reuse  strategies 
would  be  developed  and  implemented  at  this 
level . 

The  Domain  Management  Office  would 
maintain  the  domain  knowledge  and 
associated  components.  To  successfully 
incorporate  reuse  into  systems  acquisition, 
this  office  would  conduct  and  refine  domain 
analyses,  develop  and  maintain  the  domain 
model,  and  manage  resources  (library, 
personnel,  funding  allocations,  software 
tools). 

According  to  the  DoD  Software  Reuse 
Vision  and  Strategy,  software  development 
will  become  both  domain-specific  and 
library-based.  These  libraries  would  not 
only  be  tailored  to  the  domain  at  hand,  but 
be  driven  and  built  around  the  domain 
model  and  software  architecture.  The 
domain-specific  library  would  focus  not  just 
on  components,  but  also  on  the  relationships 
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Domain  Manaaement 


Figure  3 

The  Reuse  Acquisition  Planning  Structure 


that  exist  between  these  components'^.  The 
components  include  all  levels:  domain 
model,  software  architecture,  product 
design,  and  implementation  components,  as 
well  as  requirements  and  other  conceptual 
models  to  support  prototyping.  In  addition 
to  these  components,  the  library  would 
contain  reuse  metrics  to  evaluate  technical, 
schedule,  and  cost  considerations  for 
assessing  reuse  effectiveness.  To  maintain 
the  domain  model  and  the  software 
architecture,  the  Domain  Management 
Office  would  interact  with  both  the  Using 
Organization  and  the  Program  Office  (See 
Figure  3,  Reuse  Acquisition  Planning 
Structure). 

The  Domain  Management  Office  and  the 
Using  Organization  together  develop  high- 
level  requirements  to  fulfill  the  users 
needs/system.  Cost,  benefit  and  risk  trade¬ 
offs  are  made  by  prototyping  the 
requirements  using  existing  high-level 
components  (domain  model  and  software 
architecture)  from  the  supporting  library. 
Output  from  this  high-level  requirements 
analysis  is  incorporated  into  the  domain 
model.  This  allows  the  requirements  to  be 
managed  at  the  domain  level  in  addition  to 
the  system  level  as  they  are  now.  In  turn, 
requirements  from  each  system  developed 
would  be  mapped  into  the  domain  to  which 
the  system  belongs.  With  reusable 
components  managed  at  the  domain  level, 
the  components  can  be  mapped  centrally  to 
various  systems  in  the  domain**. 

The  Domain  Management  Office  and  the 
Program  Office  together  would  refine  the 
system’s  requirements  through  additional 
prototyping,  using  low-level  (designs  and 
implementation  components)  components. 
Output  from  this  system  requirements 
analysis  would  be  incorporated  into  the 
software  architecture  for  the  particular 
domain.  Reusable  components  managed  at 


the  domain  level  would  be  mapped  to  the 
various  systems  via  the  domain-specific 
library**.  Thus,  the  software  architecture  is 
maintained  by  the  Domain  Management 
Office  with  support  from  the  Program 
Office. 

Prototyping  systems  by  using  existing 
components  would  be  accomplished  at  two 
levels.  User  requirements  can  be  prototyped 
initially  at  the  domain  model/software 
architecture  level.  The  user  and  system 
requirements  can  be  further  refined  by 
prototyping  using  implementation 
components.  This  process  allows  not  only 
quicker  validation  of  requirements  and  a 
timely  determination  of  feiisibility,  but  can 
provide  early  detection  of  eiTors  and  benefit 
other  systems  being  developed  within  the 
domain. 

The  Domain  Management  Office  would 
accumulate  technical,  schedule  and  cost 
reuse  metrics  collected  by  the  Program 
Offices,  and  make  them  available  to 
developers  and  other  acquisition  managers  of 
systems  within  the  domain.  The  technical, 
schedule,  and  cost  baselines  for  a  new 
system  could  be  determined  by  using  actual 
statistics  from  programs  within  the  same 
domain.  Acquisition  and  Reuse  Plans  as 
well  as  cost  estimates  would  be  more 
accurate. 

The  domain  management  function  is  a 
somewhat  revolutionary  concept.  The 
current  acquisition  organizations  may  be 
threatened  by  perceived:  (1)  loss  of 
control/authority/autonomy;  (2)  increased 
risk  (technical/cost/schedule)  caused  by 
dependence  on  "someone  else’s  products"; 
and  (3)  reductions  in  program  budgets  due 
to  the  requirements  for  increased  support  for 
reuse  at  the  domain  management  level. 

In  actuality,  this  change  is  inevitable  and 
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already  occurring.  DoD  budgets  are 
shrinking.  DoD-level  organizations  have 
been  created  to  eliminate  development 
duplication  among  the  services  and 
consolidate  common  functions.  Perhaps 
what  remains  is  the  decision  regarding 
whether  acquisition  executives 
(PEOs/DACs)  are  reorganized  to  be  "dual- 
hatted"  for  program  and  specific  lead 
domain  function  responsibilities,  or  that 
PEOs/DACs  co-exist  with  the  Domain 
Management  Offices.  It  is  suggested  that 
PEOs/DACs  retain  lead  responsibilities  and 
that  the  domain  management  function  is 
matrixed  to  their  PEO/DAC  organizations. 


Figure  4,  A  Comparison  of  Reuse  Roles  and 
Responsibilities,  provides  a  summary  of  how 
reuse  might  be  impacted  by  the  concept  of 
the  domain  management  function.  It 
portrays  where  shifts  in  responsibilities 
would  occur  when  moving  from  the  current 
acquisition  structure  to  the  reuse  acquisition 
planning  structure.  The  chart  depicts  how 
we  see  significant  reuse  activities  managed 
under  the  current  acquisition  structure  and 
how  we  expect  they  would  be  managed 
under  our  proposed  structure.  The  "ad  hoc" 
approach  is  replaced  by  an  organized 
structure  which  assures  a  repeatable  and 
effective  process. 


RESPONSIBILITY 


Aclivity 

Reu.se  Applied  to  the 

Domain  Management 

Current  Acquisition  Structure 

Acquisition  Structure 

(1)  Domain  analysis 

(1)  PEO/DAC 

(1)  Domain  Management  Office 

(2)  Reuse  budget 

(2)  PEO/DAC 

(2)  Domain  Management  Office 

(3)  Cost  estimate 

(3) 

(3) 

-  For  reuse 

-  PEO/DAC 

-  Domain  Management  Office 

-  For  program 

-  PEO/DAC 

-  PEO/DAC 

(4)  Risk  assessment 

(4) 

(4) 

-  For  reuse 

-  PEO/DAC 

-  Domain  Management  Office 

-  For  program 

-  PEO/DAC 

-  PEO/DAC 

(5)  Program  strategy 

(5) 

(5) 

-  Overall 

-  PEO/DAC 

-  PEO/DAC 

-  Reuse 

-  PEO/DAC 

-  Domain  Management  Office 

-  Library 

(6)  Ad  Hoc 

(6)  Domain  Management  Office 

(6)  Reuse  component  management 

-  Upgrades 

-  Maintenance 

(7)  Ad  Hoc 

(7)  Domain  Management  Office 

(7)  Training,  tools,  policies 

(8)  Ad  Hoc 

(8)  Domain  Management  Office 

(8)  Data  ba.se  creation,  maintenance, 
improvement 

Figure  4 

A  Comparison  of  Reuse  Roles  and  Responsibilities 
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APPLICATIONS 

Under  this  reuse  acquisition  planning 
structure,  the  proposed  Domain  Management 
Office,  Program  Executive  Officer  and  the 
Program  Manager  can  utilize  tools  and 
templates  from  the  CARDS  Acquisition  and 
Direction  Level  Handbooks  to  incorporate 
software  reuse  into  their  respective  duties. 
Software  reuse  guidance  is  presented  by 
providing  methods,  examples, 
recommendations  and  techniques  to 
implement  various  reuse  strategies 
throughout  the  acquisition  life  cycle.  The 
implications  and  affects  of  software  reuse  on 
the  technical,  management,  cost,  schedule, 
and  risk  aspects  of  a  program/system  during 
the  acquisition  process  is  the  foundation  of 
both  documents.  The  domain  manager  can 
use  this  guidance  to  develop  plans  to 
manage  reuse  across  their  systems  and  to 
reach  the  goals  outlined  in  the  DoD 
Software  Reuse  Vision  and  Strategy 
document.  The  Program  Executive  Officer 
can  use  the  information  to  assist  in 
incorporating  software  reuse  into  the  initial 
planning  stages  of  an  acquisition,  as  well  as 
at  critical  points  within  the  acquisition  life 
cycle.  The  options  allow  the  executive  to 
gain  the  greatest  benefits  from  software 
reuse  while  optimizing  the  use  of  shrinking 
resources.  Program  Managers  and  their 
support  personnel  (such  as  Contracting 
Officers  and  Administrators,  procurement 
attorneys,  and  program  control  personnel) 
can  utilize  the  guidance  to  incorporate 
software  reuse  into  all  phases  of  the 
acquisition  life  cycle,  from  concept 
exploration  to  Post  Deployment  Software 
Support  (PDSS)  and  from  planning  the 
acquisition  strategy  through  awarding  the 
contract  to  managing  the  effort  and 
follow-on  support. 

Some  examples  of  these  tools  are:  Figure  5, 
Reuse  Acquisition  Process  Steps;  Figure  6, 


Reuse  Considerations;  and  Figure  7, 
Common  Functionalities  Across  Program. 

Figure  5,  Reuse  Acquisition  Process  Steps, 
takes  the  executive  through  the  typical  steps 
in  the  acquisition  process  and  identifies 
specific  reuse  activities  which  should  be 
accomplished  in  each  step.  The 
parenthetical  references  are  to  the  sections 
within  the  Acquisition  Handbook  where  each 
topic  is  discussed  extensively.  As  an 
example,  columns  2,  "Initiate  Program 
Direction  and  Acquisition  Planning" 
identifies  activities  the  executive  should 
assure  are  either  addressed  and  resolved 
prior  to  moving  to  the  next  step  or,  at  least, 
are  provided  a  milestone  for  completion  in 
the  future. 

Figure  6,  Reuse  Considerations,  provides 
specific  issues  that  must  be  considered  by 
the  executive.  For  instance,  column  3, 
"Initial  Investment’,  shows  what  must  be 
considered  when  first  applying  reuse 
concepts.  These  items  are  not  all  inclusive 
and  will  be  expanded  and  improved  upon  as 
more  experience  is  gained. 

Common  Functionalities  Across  Programs, 
Figure  7,  provides  a  methodology  for 
assessing  commonality  of  functions  across 
programs  within  a  particular  functional  area. 
It  can  also  be  used  across  agencies,  for 
example,  to  identify  common  avionics 
functionalities  among  all  military  services. 
This  perspective  enables  the  executive  to  not 
only  assess  whether  reuse  may  be  practical 
but  also,  with  supporting  data,  to  identify 
whether  a  lead  program,  or  programs  ,  can 
be  identified  for  various  functions.  It  also 
facilitates  development  of  inputs  for  cost 
effectiveness  assessments  by  categorizing  the 
degree  of  expected  commonality  and 
potential  cost  avoidance.  This  particular 
example  addresses  intelligence  training 
systems. 
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Figure  5 

Reuse  Acquisition  Process  Steps 
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FUNCTION 

PROGRAMS 

Current  Program 

Program  HI 

Program  #3 

Program  #4 

Message  Handling 

High  Commonality 

Good  Commonality 

High  Commonality 

High  Commonality 

Data  Management 

Good  Commonality 

Good  Commonality 

Good  Commonality 

Good  Commonality 

DBMS 

Good  Commonality 

Good  Commonality 

Good  Commonality 

Good  Commonality 

Software  Architecture 

High  Commonality 

Good  Commonality 

Good  Commonality 

High  Commonality 

User  Interface 

High  Commonality 

Good  Commonality 

High  Commonality 

Some  Commonality 

Figure  7 

Common  Functionalities  Across  Programs 


CONCLUSION 

While  developing  the  concepts  described  in 
this  paper,  we  have  worked  with  the 
organizations  within  DoD  which  are 
sponsoring  and  supporting  reuse.  The 
Management  Issues  Working  Group 
(MIWG),  which  includes  representatives 
from  OSD,  all  the  services,  DISA,  Defense 
Logistics  Agency  and  the  National  Security 
Agency,  has  reviewed  and  commented  on 
related  work.  Our  approach  has  been  to 
work  with  these  groups  and  others  to  obtain 
the  widest  possible  perspectives  and  ideas. 
We  have  also  received  support  from  industry 
in  review  and  critique  of  our  work.  These 
organizations,  along  with  the  acquisition  and 
software  reuse  communities,  must  work 
together  to  implement  these  concepts  to 
reach  the  cost  and  schedule  benefits  of 
software  reuse. 
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ABSTRACT 


In  1992  - 1993,  a  ten-month  Senior  Acquisition  Course  was  offered  for  the  first  time 
to  a  select  group  of  senior  military  and  civilian  acquisition  professionals.  Growing 
out  of  legislation  enacted  by  Congress  which  also  created  the  Defense  Acquisition 
University,  the  course  is  designed  to  provide  senior  acquisition  professionals  with  an 
educational  experience  which  is  comparable  and  equivalent  to  that  of  a  senior 
service  school  or  war  college.  This  paper  provides  an  analysis  of  the  first  year's 
experience  with  that  new  course,  conducted  at  the  Industrial  College  of  the  Armed 
Forces.  It  examines  the  genesis  of  the  course,  needs  asessment,  course  content,  and 
preliminary  results  of  the  first  year  offering.  In  addition,  the  paper  analyzes  students 
selected  for  the  course  in  terms  of  make-up,  experience,  and  credentials.  Finally,  it 
briefly  examines  possible  future  directions  for  both  the  senior  course  and  students 
likely  to  complete  it  in  the  evolving  acquisition  career  workforce  environment.  It  is 
expected  that  this  analysis  will  provide  important  insights  for  policy  makers, 
practitioners,  and  educators  concerned  with  shaping  needs  for  senior  level 
acquisition  education  as  well  as  for  the  intermediate  education  levels  which  precede 
it. 


The  views,  opinions,  and  /  or  conclusions  in  this  paper  are  those  of  the  authors  and 
should  not  be  construed  as  an  official  position  of  the  Department  of  Defense,  the 
Industrial  College  of  the  Armed  Forces,  or  any  other  government  agency. 


91 


INTRODUCTION 

On  June  16,  1993,  a  group  of  37  highly 
select  senior  military  officers  and  civilian 
officials  graduated  from  the  first  senior 
acquisition  course  with  full  status 
comparable  to  that  of  any  senior  service 
college  in  the  Department  of  Defense. 
This  new  Senior  Acquisition  Course, 
conducted  at  the  Industrial  College  of 
the  Armed  Forces  (ICAF),  marks  the  first 
offering  of  its  kind  to  senior 
professionals  in  the  critical  business  of 
defense  acquisition  management. 
Borne  out  of  legislation  which  also 
created  the  Defense  Acquisition 
University,  the  course  represents  a 
turning  point  in  providing  an  executive- 
level  educational  program  for  those 
select  acquisition  leaders  and  managers 
who  may  be  destined  to  rise  to  general 
and  flag  officer  or  senior  executive 
service  positions  in  the  acquisition 
workforce. 

The  genesis  of  this  new  course,  along 
with  an  analysis  of  the  work  that  went 
into  student  needs  asessment, 
curriculum  design,  and  the  first  year's 
experience  in  conducting  it,  provide 
important  insights  for  policymakers, 
practitioners,  and  educators  concerned 
with  shaping  future  directions  in 
executive  level  acquisition  programs. 

BACKGROUND 

Through  the  latter  half  of  the  1980s,  a 
series  of  comprehensive  studies  were 
conducted  at  the  Defense  Systems 
Management  College  aimed  at 
enhancing  the  professional  acquisition 
career  field.  Impelled  in  part  by  those 
studies.  Congress  enacted  legislation  (10 
use  1746)  in  1990  which  established  a 
new  Defense  Acquisition  University 
(DAU)  to  oversee  the  educational  needs 
and  programs  of  professionals  in  that 
workforce.  In  addition,  the  law  directed 
DOD  to  establish  "a  senior  course  as  a 
substitute  for,  and  equivalent  to, 
exisiting  senior  professional  military 
education  (PME)  school  courses. 


specifically  designed  for  personnel 
serving  in  critical  acquisition  positions." 

Responding  to  this  requirement,  the 
Under  Secretary  of  Defense  (Acquisition) 
[USD(A)]  on  July  1,  1991,  announced  his 
choice  of  the  Industrial  College  of  the 
Armed  Forces,  which  currently  met  all 
statutory  requirements  for  senior  PME, 
to  take  on  this  enhanced  educational 
mission  for  senior  members  of  the  DOD 
acquisition  workforce.  The  Chairman, 
Joint  Chiefs  of  Staff,  under  whose 
auspices  ICAF  falls,  joined  in  agreeing  to 
USD(A)'s  oversight  of  the  acquisition 
content  of  the  ICAF  curriculum. 

Interestingly,  this  new  task  built  upon 
an  historic  part  of  the  ICAF  mission 
which  had  its  roots  in  the  College's 
original  1924  charter  as  the  Army 
Industrial  College,  (re-established  as  the 
present  day  Industrial  College  in  1948) 
War  Department  General  Order  No.  7, 
1924,  established  the  purpose  of  the 
institution  to  include  educating  officers 
in  the  "supervision  of  procurement"  of 
military  equipment.  Nearly  70  years 
later,  the  contemporary  ICAF  mission 
similarly  calls  for  providing  a  senior 
education  with  "special  emphasis  on 
materiel  acquisition." 

Thus  chartered  by  the  USD(A),  ICAF 
began  the  process  of  designing  and 
developing  the  new  Senior  Acquisition 
Course  to  be  offered  in  Academic  Year 
1992-  1993. 

NEEDS  ASSESSMENT 

The  standards  for  all  senior  PME  courses 
are  established  by  the  Military 
Education  Policy  Document  (MEPD)  CM 
344-90,  issued  by  the  Chairman  of  the 
Joint  Chiefs  of  Staff.  In  general,  senior 
PME  is  10  months  in  duration  and  is  to 
consist  of  common  core  requirements 
focused  at  the  national  strategic  level, 
incorporating  both  national  and 
international  considerations  as  they 
affect  the  formulation  of  national 
security  policy.  Each  college  is  expected 
to  augment  studies  in  national  security 
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with  studies  associated  with  its  unique 
mission.  In  the  case  of  ICAF  and  the 
Senior  Acquisition  Course,  this  translates 
into  an  emphasis  on  the  resources  which 
support  national  strategy  including  the 
means  whereby  we  acquire  them  and 
the  responsiveness  of  the  national  and 
global  industrial  base  which  provides 
those  resources. 

Building  upon  that  basic  foundation,  a 
team  was  assembled  to  systematically 
assess  the  special  educational  needs  of 
senior  acquisition  professionals  and 
then  craft  a  course  responsive  to  those 
requirements.  Various  individuals  and 
organizations  played  key  roles  in  that 
process  in  the  months  that  followed. 

Prompted  by  a  general  officer  level 
working  session  convened  in  the  Air 
Force,  the  Joint  Logistics  Commanders 
(JLC)  formed  a  group  expressly  to 
consider  the  educational  needs  of  senior 
acquisition  professionals.  With 
representatives  of  each  Service,  the 

?iroup  worked  for  some  6  months  to 
ormulate  a  joint  recommendation  to 
the  Service  Acquisition  Executives  and 
the  ICAF  on  the  content  for  the  new 
Senior  Acquisition  Course.  Most  notable 
in  their  final  report  was  agreement  on 
11  major  areas  for  which  the  JLCs 
believe  senior  leaders  and  managers  in 
the  acquisition  field  must  have  critical, 
analytical  competencies.  The  list  is 
particularly  instructive,  not  so  much  in 
terms  of  subject  area,  but  rather  for  the 
breadth  and  national  level  focus  it 
reveals: 


At  ICAF,  a  team  was  fo:  ned  to 
incorporate  these  and  other  inputs  into 
the  design  of  the  new  course.  A 
curriculum  bcommittee  was  chartered 
to  design  the  course*  and  a  prototype 
offering  was  conducted  in  Academic 
Year  1991  -  1992  consisting  of  students 
with  extensive  acquisition  credentials 
and  experience  identified  from  within 
the  existing  ICAF  class  that  year.  Lessons 
learned  from  that  experience,  coupled 
with  other  internal  and  external 
analyses  (including  ongoing  studies  of 
the  needs  of  executive  decision  makers 
and  leaders  by  the  Army  Research 
Institute)  and  the  JLC  needs  assessment, 
were  melded  to  craft  the  purpose, 
objectives,  and  content  of  the  new 
course  to  be  offered  in  Academic  Year 
1992  -  1993.  The  JLCs  favorably 
reviewed  these  plans  in  the  spring  of 
1992. 

COURSE  PURPOSE  &  OBJECTIVES 

In  consultation  with  the  USD(A),  the 
purpose  of  the  new  course  was  stated  in 
these  words; 

The  Senior  Acquisition  Course  is 
designed  to  prepare  acquisition 
professionals  to  function 
effectively  and  efficiently  in  a 
complex  and  dynamic  geopolitical, 
economic,  social,  and  military 
environment. 

The  Senior  Acquisition  Course; 

-  examines  acquisition  concepts 
and  practices  within  a  context  of 
resource  constraints  that  mandate 
comprehensive,  reasoned  linkages 
between  national  strategy  and 
national  resources 


-  management  of  risk 

-  fiscal  &  monetary  policies 

-  environmental  considerations 

-  decisionmaking 

-  international  acquisition  issues 

-  ethics 

-  national  policy  /  industrial  base 
-Congressional  relations 

-  evolution  of  acquisition 

-  global  economy  /  competition 

-  industry  motivation 


-  provides  a  unique  postgraduate, 
executive  level  education  that 
integrates  a  special  emphasis  on 
acquisition  management  with 
studies  in  resources  management 
and  national  security  strategy. 
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Extrapolating  from  that  broad  charter,  a 
set  of  specific  objectives  were  crafted  for 
the  course.  Those  objectives  call  for 
graduating  senior  acquisition  course 
students  wno  will... 

-  be  broadly  educated  strategic 
thinkers  who  understand  the  role 
of  acquisition  within  the  context  of 
the  resources  component  of 
national  security  strategy  and  are 
competent  to  make  effective 
acquisition  decisions  and  policies  in 
a  complex  and  rapidly  changing 
national  security  environment. 

-  receive  a  premier  education  in 
national  security  resources 
management  with  special 
emphasis  on  acquisition,  joint 
logistics,  and  mobilization. 

-  receive  a  program  of  Joint 
Professional  Education  that  will 
prepare  them  to  operate  in  a  joint 
and  combined  environment  and  to 
think  strategically  from  a  joint  and 
combined  perspective. 

-  be  integrated  into  the  ICAF 
student  body,  composed  of  a 
balance  of  operators  (30%), 
resource  managers  and  technical 
personnel  (50%),  and  civilians 
(20%). 

-  be  exposed  to  national  and 
international  security  communities 
and  to  evolving  security  resource 
management  issues  as  they  relate 
to  acquisition. 

-  be  fully  prepared  to  assume  SES 
and  general/flag-level  acquisition 
positions 

The  resultatant  course  was  designed  to 
fulfill  those  objectives  and  to  meet  the 
needs  of  the  acquisition  customer 
community  and  the  nation  at  large. 

THE  10  MONTH  SENIOR  COURSE 


The  senior  PME  acquisition  course  core 
curriculi'm  has  been  divided  into  two 
semesters  (see  Figure  1).  The  core 
content  is  designed  to  meet  statutory 
requirements  for  senior  PME 
equivalency  and  to  build  the  conceptual 
foundation  for  the  study  of  national 
resource  management  policy.  It  includes 
the  integrated  and  interdisciplinary 
study  of  national  resource  management 
through  courses  in  political  science, 
economics,  strategic  decisionmaking, 
strategy  and  warfare,  history  and 
regional  studies,  elements  of  national 
industrial  power,  joint  logistics, 
acquisition  and  mobilization  / 
reconstitution. 

The  curriculum  consists  of  courses 
presented  in  an  integrated  mix  of 
seminars  and  lectures.  With  a  focus  on 
active  learning,  it  relies  heavily  on 
historical  case  studies  --  from  the 
Peloponnesian  Wars  through  the  Gulf 
War  -  which  are  used  to  understand 
their  political  background,  strategic 
significance,  the  policy-strategy- 
resource  relationship,  the  evolution  of 
national  strategies,  the  acquisition  and 
mobilization  of  national  resources,  and 
their  relevance  to  today's  issues.  Case 
studies  are  used  extensively  throughout 
the  course.  The  case  studies  are 
complemented  by  intensive  student 
reading,  writing,  and  oral 
presentations;  classroom  analyses;  on 
site  field  trips;  and  a  domestic  and 
internatio.ial  field  study  program. 

Fall  Semester:  The  fall  semester 
introduces  students  to  the  concept  of 
National  Security  Strategy  -  the  art  and 
science  of  developing  and  using  the 
political  /  diplomatic,  economic,  and 
psychological  powers  of  a  nation, 
together  with  its  armed  forces,  during 
peace  and  war,  to  secure  national 
objectives.  This  semester  lays  the 
groundwork  for  the  examination  of  the 
management  of  national  resources  in 
the  spring  semester. 

The  focus  is  on  the  global  dimension  of 
U  S.  national  strategy  and  the 
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interdisciplinary  nature  of 
decisionmaking  at  the  national  level. 
The  domestic  political  system  for  the 
formulation  of  national  security  policy 
and  the  role  of  the  United  States  in  the 
international  system  are  examined.  The 
importance  of  the  economic  component 
of  national  power  is  stressed,  as  well  as 
its  impact  on  the  formulation  and 
implementation  of  national  policy  and 
grand  strategy. 

Spring  Semester:  The  spring  semester 
confronts  major  issues  associated  with 
acquiring  and  managing  national 
resources  in  support  of  national 
strategy.  The  curriculum  focuses  on  the 
sources  of  national  wealth  as  well  as  the 
decisionmaking  process  used  to  convert 
those  sources  into  the  goods  and 
services  which  underwrite  our  national 
security  objectives  with  national  power. 
The  semester  is  built  around  an 
interdisciplinary  curriculurr.  that 
continues  the  approach  characteristic  of 
studies  in  the  fall. 

Figure  2  provides  a  framework  for  the 
second  semester  curriculum  Withm 
that  overall  framework,  students 
examine  the  interaction  between 
national  strategy,  the  elements  of 
national  power,  the  nation's  resources 
and  the  acquisition,  allocation,  and 
management  of  those  resources  to 
support  national  strategy.  Each  student 
also  selects  one  of  sixteen  industries  to 
allow  this  interaction  to  be  understood 
in  some  depth. 

Enhanced  Acquisition  Management 
Studies.  Class  sessions  have  been 
integrated  throughout  the  10-month 
curriculum  to  emphasize  the  strategic 
dimensions  of  acquisition  topics. 
Seminars,  lectures,  and  case  studies  have 
been  designed  to  challenge  the  students 
to  assess  current  acquisition  policy  and 
practice  in  the  context  of  National 
Security  Strategy  with  an  emphasis  on 
"Acquisition  Management  in  a 
Changing  World."  In  addition,  thirty 
classes  have  been  developed  for  Senior 
Acquisition  Course  stucients.  These 


classes  assess  the  conditions,  influences, 
and  the  stakeholders  which  determine 
acquisition  policy,  and  encourage  or 
hinder  alternate  paradigms. 

A  broad  range  of  national  security  and 
acciuisition  experts  (and  critics) 
adciressed  the  students.  Speakers  were 
drawn  from  senior  level  decision  makers 
such  as  Gen.  Colin  Powell,  Chairman  of 
the  Joint  Chiefs  of  Staff,  the  Chiefs  of 
each  of  the  Services;  Dr.  David  Chu, 
formerly  of  OSD;  Mr.  James  Woolsey,  , 
Director  of  the  CIA;  former  Secretary  of 
the  Navy  Lawrence  Garrett;  Gen.  Tuttle, 
former  Commander  of  AMC;  prominent 
industrialists  such  as  Mr.  Norman 
Augustine  (Martin  Marietta),  and  Mr. 
Michael  Bonsignore  (Honeywell);  other 
stakeholders  such  as  Senator  Bingamon 
(NM);  key  figures  in  the  media  such  as 
Mr.  David  Gergen  (U  S.  News  and  World 
Report);  experts  from  academia  such  as 
Dr.  Barry  Bosworth,  Dr.  Ronald  Fox,  and 
Dr.  Jacques  Gansler;  and  members  of 
the  international  acquisition  community 
such  as  Gen.  Alain  Cremieux  of  the 
French  DGA. 

Advanced  Studies.  In  addition,  each 
student  selected  three  elective  courses 
for  intensive  study  (twelve-2-hour  class 
sessions  in  each).  A  wide  array  of  topics 
covering  all  aspects  of  acquisition 
management  were  offered  for 
advanced  study.  These  studies  are 
designed  to  enhance  and  complement 
the  core  curriculum  with  emphasis  on 
the  specific  needs  and  interests  of  each 
student.  All  advanced  studies  maintain 
academic  rigor  through  student  oral 
reports,  research  papers  or  essays. 
Acquisition  related  courses  included: 
Government-Industry  Relations; 
Professional  Ethics  for  Public  Officials; 
Senior  Leadership-Power  &  Politics; 
Project  Management  Using 
Microcomputers;  Planning,  Programing 
and  Budgeting  Systems;  Advanced 
Defense  Contracting;  Advanced  Seminar 
in  Selected  Acquisition  Topics;  Business 
Ethics;  Models  of  Pro  gram 
Management;  International  Acquisition, 
etc. 
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Highest  degree  held: 


Research.  All  acquisition  students 
undertook  a  major  research  project  and 
produced  a  fully  documented,  scholarly 
paper.  Topics  of  timely  interest  were 
solicited  from  throughout  the 
acquisition  community  and  the  national 
security  community  at  large  for  student 
research.  Each  project  was  overseen  by 
a  faculty  research  advisor. 

Faculty  Advisor.  Each  sr.  'ior  acquisition 
student  was  assigned  a  Primary  Faculty 
Advisor  (PFA)  who  is  a  Professor  of 
Acquisition.  Students  are  required  to 
coordinate  their  advanced  study  choices 
and  research  topics  with  their  PFA  who 
approves  the  selections,  ensuring  the 
selections  complement  the  student's 
individual  goals  and  career  needs. 


STUDENTS 

The  Services,  Defense  Agencies,  and  the 
Office  of  the  Secretary  of  Defense  (OSD) 
competitively  selected  37  senior  civilians 
and  military  officers  to  attend  the 
Academic  Year  1992  -  1993  program 
which  began  on  August  17,  1992.  The 
breakdown  is  shown  below: 


Mil 


Army  5 

Navy  4 

Air  Force  9 

Marines  1 


OSD/Agencies 


Civ  Total 
6  11 

5  9 

1  10 

1 

6  6 


Totals  19  18  37 


Beyond  the  numerical  breakout,  a 
number  of  other  descriptive  indicators 
lend  some  insight  into  the  composition, 
credentials,  and  experience  or  the  first 
class. 


doctorate  3 

master  32 

bachelor  2 

Military  vs.  civilian  &  grade  level : 

Mil  ^ 
0-5/GS-14  13  3 

0-6/GS-15  6  15 

Years  of  Acquisition  Experience: 

Range:  3  years  to  20  +  years 

Average:  10  years 

Graduates  of  DSMC: 

PMC  graduates  - 1 1  (of  37) 

What  these  numbers  alone  do  not 
reveal  is  the  rich  background  in 
experience  that  the  individuals  selected 
for  the  senior  course  bring  to  this 
program.  As  a  group,  they  include 
program  managers,  interim-level  policy 
makers,  and  practitioners  from  most 
facets  of  acquisition  management. 
Their  years  in  the  business  come  from 
stints  in  program  offices,  acquisition 
management  and  oversight  staffs,  and 
the  various  power  corridors  of 
Washington.  In  short,  these  students 
are  promising  senior  members  of  the 
professional  acquisition  workforce. 


PRELIMINARY  FINDINGS 


While  the  first  class  of  the  senior 
acquisition  course  graduated  only 
recently,  observations  and  interim 
assessments  from  throughout  the  10- 
month  program  permit  some  tentative 
though  preliminary  conclusions  about 
results. 


Gender: 

men  31 

women  6 


Credentials:  In  terms  of  experience  and 
credentials,  the  first  37  students  selected 
for  this  program  compare  favorably 
with  other  competitively  selected 
students  in  senior  PME. 


96 


Performance:  The  students  in  the  senior 
course  performed  at  a  level  on  an  equal 
par  with  other  students  enrolled  in  the 
regular  ICAF  curriculum.  Some 
competed  favorably  for  designation  as 
"distinguished  graduate." 

Depth  vs.  Breadth:  Results  are  mixed 
here,  though  favorable  in  either  case. 
Some  used  the  options  available  in 
research  and  electives  course  offerings 
to  probe  deeper  in  areas  commensurate 
with  their  entering  area  of  expertise  in 
acquisition.  Others  used  the  10-month 
program  as  an  opportunity  to  briaden 
their  horizons  in  some  new  facet  of 
acquistion  management  or  some 
broader,  related  aspect  of  national 
security  management. 

integrated  Classes:  Nearly  all  students 
and  faculty  reacted  favorably  to  having 
students  in  the  senior  acquisition  course 
fully  integrated  in  core  curriculum 
courses  with  other  ICAF  students.  The 
interaction  with  students  from 
operational,  logistical,  and  other 
support  communities  appears  to  have 
enhanced  the  acquisition  students 
debate  of  difficult  issues  within  the 
larger  context  of  which  they  are  a  part. 

Separate  Acquisition  Classes: 
Segregating  senior  acquisition  students 
only  for  their  spring  core  curriculum 
course  in  Acquisition  Management 
carries  both  strengths  and  weaknesses. 
On  the  positive  side,  senior  course 
students  are  able  to  rapidly  move  to 
sophisticated  levels  of  debate  in  a 
curriculum  specifica.Hy  tailored  to  their 
advanced  levels  of  experience. 
Conversely,  those  ICAF  students  with 
lesser  or  no  experience  in  acquisition  are 
able  to  cover  acquisition  issues  at  a  level 
more  suited  to  their  needs  At  the  same 
time,  those  students  with  lesser 
experience  are  somewhat  limited  in 
their  interaction  with  senior  acquisition 
participants  -  peers  with  whom  adult 
education  theory  suggests  a  great  deal 
of  learning  occurs.  Moreover,  senior 
course  students  in  class  by  themselves 
are  often  denied  the  tough,  challenging 


questions  those  less  familiar 
(encumbered?)  with  the  business 
sometimes  pose.  And  there  is  the 
occasional  hazard  of  segregated  senior 
course  students  "talking  acronyms"  or 
becoming  submerged  in  details  with 
one  another  to  the  detriment  of  rising 
to  the  national  level  context  of  debate 
desired. 

Civilian  Workforce  Career  Issues:  While 
members  of  the  civilian  workforce  have 
participated  in  senior  PME  in  increasing 
numbers  in  the  past  decade,  their 
participation  in  the  Senior  Acquisition 
Course  in  still  greater  numbers  raises 
several  potential  acquisition  career 
management  issues; 

-  Selection  Criteria.  What  criteria  should 
the  military  departments  and  other 
defense  agencies  use  in  competitively 
selecting  promising  senior  civilians  for 
attendance  at  this  course?  The  military 
has  a  long  history  of  stringent  PME 
selction  criteria  for  uniformed  officers 
followed  by  functional  career 
progression  through  various  levels  of 
achievement.  At  a  time  when  civilian 
career  management  for  acquisition 
professionals  has  begun  to  mirror  that 
military  process,  it  seems  critical  that 
similarly  clear  and  stringent  criteria  are 
crafted  to  sustain  the  exceptional 
quality  of  civilian  professionals  who 
participated  in  the  first  course. 

'  Career  Enhancement.  How  will  defense 
acquisition  career  managers  insure  that 
a  10-month  absence  from  the  civilian 
workplace  becomes  a  positive  step 
forward  in  a  progressive,  advancing 
civilian  career  path.  Moreover,  since  the 
Senior  Acquisition  Course  is  not  a 
mandatory  course  (no  senior  PME  course 
is  mandatory;  rather  they  are  designed 
to  be  executive-level  educational 
experiences  for  a  small,  highly  select 
number  of  potential  general  /  flag  /  SES 
contenders),  how  will  attendance  help 
assure  an  equitably  competitive 
opportunity  for  career  ascension  in  the 
civilian  acquisition  community? 
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THE  FUTURE? 


The  first  year's  experience  with  the  new 
Senior  Acquisition  Course  is  just  that  --  a 
first  experience.  It  is  a  point  of 
departure  for  refinement  and  evolution. 

Student  enrollment  is  projected  to 
increase  frcm  present  levels  to  76  by 
1995. 

Course  content  will  undoubtedly  be 
shaped  by  internal  and  external  forces 
and  the  multipolar  stakeholders  which 
comprise  the  acquisition  community  as 
well  as  those  charged  with  its  oversight. 
The  continuing  challenge  will  be  to 
refine  customer  needs,  engender 
intellectual  innovation,  foster 
enlightened  executive  acquistion 
leadership,  and  ultimately  insure 
customer  responsiveness,  both  to  the 
members  of  our  armed  forces  and  the 
nation's  taxpayers. 
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The  Senior  Acquisition  Course 
Two  Semester  Curriculum  Plan 


1ST  SEMESTER 


2ND  SEMESTER 


National  Security  Strategy 


Resourcing  National  Strategy 


FIGURE  1 
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SALVAGING  THE  DEFENSE  INDUSTRIAL  BASE 


Lawrence  Briskin,  United  States  Air  Force 


ABSTRACT 

The  United  States  defense  industrial 
base  has  been  in  various  stages  of  decline 
for  the  last  45  years.  It  has  been  part  of  the 
erosion  of  the  industrial  base  of  the  entire 
nation.  The  process  has  accelerated  in  the 
last  12  years. 

With  the  winddown  of  the  Cold  War 
and  a  final  conclusion  of  World  War  II,  as 
witnessed  by  hostilities  long  repressed  in 
Europe  and  the  former  Soviet  Union,  re¬ 
sponses  once  deemed  undesirable,  have 
suddenly  become  available. 

Appropriate  action  in  the  defense 
arena  is  doubly  critical.  The  loss  of  manu¬ 
facturing  capability  already  experienced,  is 
about  to  be  compounded  by  loss  of  manufac¬ 
turing  orders  due  to  the  drawdown. 

The  United  States  is  now  in  a  posi¬ 
tion  to  respond  with  appropriate  actions  with 
regard  to  either  the  defense  industrial  base  or 
the  entire  economy.  This  paper  examines 
some  responses  with  regard  to  both,  weigh¬ 
ing  the  pros  and  cons  of  each. 

INTRODUCTION 

"Computer  chips,  potato  chips,  what’ - 
s  the  difference?  They’re  all  chips.  One  hun¬ 
dred  dollars  of  potato  chips  and  one  hun¬ 
dred  dollars  of  computer  chips  are  both  one 
hundred  dollars."  Michael  Boskin,  Chair¬ 
man  of  the  President’s  Council  of  Economic 
Advisors,  1985  [10], 

The  United  States  has  been  in  the 
process  of  deindustrialization  since  World 
War  n.  Part  of  the  process  has  been  a  result 
of  the  end  of  the  Cold  War.  Part  has  been 
the  result  of  an  organized  campaign  by  our 
trade  competitors. 


When  World  War  II  ended  the  United 
States  was  the  unquestioned  dominant  eco¬ 
nomic  and  military  power.  It  was  natural 
that  other  nations  gain  relative  economic 
strength  as  a  result  of  their  recovery.  By 
1970  the  combatants  had  fully  recovered. 

Though  the  U.S.  still  had  trade  sur¬ 
pluses,  the  continuing  peacetime  decline  of 
American  industry  could  no  longer  be  traced 
to  the  recovery  of  the  world  from  the  war. 
The  year  1970  began  the  accelerated  decline 
in  American  industry  continuing  to  this  day. 
Our  trade  deficit  began  to  grow.  There  was 
no  growth  in  manufacturing  employment 
Meanwhile  the  American  economists  and  our 
government  continued  to  preach  the  benefits 
of  free  trade. 

A  strong  military  posture  was  neces¬ 
sary  because  of  the  continuing  Cold  War.  A 
strong  defense  industrial  base  was  the  natu¬ 
ral  adjunct.  With  the  winddown  of  the  Cold 
War  and  the  psychological  conclusion  of 
World  War  11,  following  the  dissolution  of 
the  Soviet  Union,  remedies  considered  ques¬ 
tionable  have  suddenly  opened. 

The  U.S.  economy  is  closely  inter¬ 
twined  with  that  of  the  world.  Therefore  it  is 
necessary  to  treat  the  issues  from  two  asp¬ 
ects,  the  relationship  of  the  world  economy 
to  that  of  the  U.S.,  and  the  relationship  of 
the  civilian  U.S.  economy  to  the  defense 
industry. 

PROBLEMS  IN 

THE  GENERAL  ECONOMY 

The  decline  of  the  general  economy 
is  easily  measured.  In  1970  the  population 
of  the  United  States  was  about  200  million. 
There  were  about  24  million  persons  em¬ 
ployed  in  manufacturing.  In  1990  the  popu¬ 
lation  of  the  United  States  was  about  250 
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million.  There  were  still  about  24  million 
persons  employed  in  manufacturing  [5].  In 
other  words,  the  population  had  grown  by  25 
percent  but  the  number  of  people  employed 
in  manufacturing  has  remained  unchanged. 
Figure  1  illustrates.  Thus,  net  goods  to  sup¬ 
ply  the  50  million  person  population  growth 
are  being  imported.  This  represents  goods  to 
supply  20  percent  of  the  current  population. 
There  has  been  a  productivity  improvement 
in  U.S.  manufacturing,  but  not  nearly  enough 
to  account  for  the  no  job  growth  in  manufac¬ 
turing. 

If  manufacturing  employment  had 
grown  along  with  population,  there  would  be 
25  percent  or  6  million  more  people  employ¬ 
ed  in  manufacturing.  The  additional  unem¬ 
ployed  also  represent  thousands  of  factories 
which  no  longer  exist. 

It  is  true  that  there  has  been  sub¬ 
stantial  growth  in  the  non-manufacturing 
sector,  but  jobs  in  nursing  homes,  fast  food. 


and  discount  houses  pay  little  more  than  the 
minimum  wage.  They  contribute  nothing  to 
the  industrial  base. 

Compounding  the  long  term  problem 
is  the  current  drawdown  of  the  military 
services.  Not  only  are  hundreds  of  thou¬ 
sands  of  persons  being  separated  from  the 
services,  but  production  of  new  weapons 
systems  is  being  slowed  or  cancelled,  af¬ 
fecting  the  defense  industrial  base  in  terms 
of  factories  and  people. 

What  do  we  do  now? 

THE  FOLLIES  OF  ADAM  SMITH 

By  dividing  the  problem  into  compo¬ 
nents  it  is  possible  to  deal  with  each.  The 
long  term  civilian  problem  may  require  dif¬ 
ferent  solutions  than  those  for  the  defense 
industrial  base. 

Problems  with  the  general  economy 
may  be  traced  to  American  economists  in 


Figure  1  -  Employment  and  Trade  Deficit 
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their  blind  adherence  to  the  teachings  of 
Adam  Smith.  Whatever  the  merits  of  Smith¬ 
’s  arguments  with  respect  to  competition 
within  a  nation,  they  have  serious  faults  in 
today’s  world  of  international  trade. 

Smith  developed  his  argument  by 
analogy.  If  a  tailor  and  shoemaker  are  each 
making  both  shoes  and  clothing  for  then- 
own  need,  they  would  be  wealthier  if  the 
shoemaker  made  shoes  only,  the  tailor 
clothes  only  and  they  traded.  The  implicit 
assumptions  are,  that  they  were  fully  em¬ 
ployed  prior  to  manufacture  for  trade,  that 
both  need  and  want  more  goods,  and  that  the 
science  of  economics  will  assist  them  in 
decision  making. 

Under  the  first  assumption,  if  the 
tailor  increases  the  manufacture  of  clothing, 
he  must  reduce  the  manufacture  of  shoes. 
When  he  reduces  the  manufacture  of  shoes, 
this  leaves  an  opening  for  the  shoemaker  to 
sell  him  shoes. 

Under  the  second  assumption,  the  tai¬ 
lor  would  like  to  have  more  shoes  and/or 
clothing.  He  doesn’t  have  as  much  as  he 
wants  or  needs. 

Lastly,  all  goods  are  in  short  supply 
and  the  science  of  economics  will  help  us 
make  and  distribute  them. 

Even  if  these  assumptions  were  real¬ 
istic  for  the  tailor  and  the  shoemaker,  they 
need  not  be  true  for  national  economies. 

HOW  LEVEL  IS  THE  PLAYING 
FIELD? 

Do  labor  rates  count?;  Many  coun¬ 
tries  that  have  large  trade  surpluses  with  the 
United  States,  have  much  lower  labor  rates. 
These  include  Mexico,  China,  Indonesia, 
Brazil,  Thailand,  and  many  others. 

Standard  theory  recognizes  that  every 
element  of  cost  contributes  to  the  ultimate 
selling  price  and  competitiveness  of  a  prod¬ 
uct.  Because  U.S.  labor  costs  are  substan¬ 
tially  higher  than  in  these  nations,  American 


goods  could  never  compete  on  a  head-to- 
head  basis,  if  the  productivity  in  the  United 
States  is  not  substantially  higher  than  in 
these  nations. 

When  faced  with  this  issue  the  typi¬ 
cal  textbook  response  is  that  higher  Ameri¬ 
can  productivity  more  than  compensates  for 
lower  foreign  wages.  Unfortunately  this 
analysis  is  faulty. 

Taking  Mexico  as  an  example,  it  is 
true  that  American  productivity  is  higher 
than  Mexican.  If  Mexicans  were  only  80 
percent  as  productive  as  Americans,  and 
wages  were  the  same,  the  textbook  response 
would  be  correct. 

However  Americans  are  paid  substan¬ 
tially  more  than  Mexicans.  If  Americans 
were  paid  four  times  the  Mexican  wages, 
then  Mexican  goods  would  cost  significantly 
less  than  American.  Computing  costs: 

($/hr)/(unit/hr)  =  ($/unit) 

In  the  United  States: 

$8.00/10  =  $.80 

In  Mexico: 

$2.00/8  =  $.25. 

Mexican  goods  cost  25/80  or  31.25 
percent  of  American  goods.  Under  these  cir¬ 
cumstances  American  industry  has  been 
doing  the  rational  thing,  moving  wholesale 
to  Mexico  and  other  Third  World  nations. 

Japan  is  another  critter.  The  Japa¬ 
nese  through  the  Ministry  of  International 
Trade  and  Industry  (MITI)  systematically 
encouraged  the  formation  of  keiretzus.  Keire- 
tzus  are  industrial  combinations  similar  to 
what  we  call  trusts.  Keiretzus  would  be 
illegal  in  the  United  States.  These  large 
combinations  then  compete  with  individual 
American  companies  [12].  The  situation  be¬ 
comes  American  Computing  Company  XX 
vs  Japan  Inc.,  hardly  an  equal  competition. 
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DOES  FREE  TRADE  EXIST 

Many  economists  still  hold  to  the 
doctrine  of  free  trade.  They  cling  to  its 
implementation  through  the  General  Agree¬ 
ment  on  Tariffs  and  Trade  (GATT).  Unfor¬ 
tunately  the  GATT  system  has  been  engulfed 
in  a  daisy-chain  of  negotiations.  The  GATT 
consists  of  over  200  treaties  leaving  room 
for  negotiation,  delay,  and  evasion.  The 
Uruguay  round  of  GATT  negotiations  has 
been  going  on  for  six  years. 

Recently  the  United  States  and  Cana¬ 
da  entered  into  a  Free  Trade  Agreement 
(FTA).  The  FTA  is  over  1000  pages  long. 
The  proposed  North  American  Free  Trade 
Agreement  (NAFTA)  with  Mexico  is  more 
than  2000  pages. 

Contrary  to  the  laissez-faire  approach 
implicit  in  free  trade,  Japan  and  others  have 
engaged  in  export  drives  deliberately  build¬ 
ing  large  trade  surpluses.  "Tn  my  time,  our 
job  was  simple,’  says  Naohiro  Amaya,  a 
former  MITI  official  and  an  architect  of 
Japan’s  postwar  economic  miracle.  ’We  had 
only  to  solve  a  single  equation’  -  how  to 
promote  exports  [8]." 

It  seems  that  free  trade  is  a  highly 
managed  process,  going  nowhere.  Free  trade 
does  not  exist.  It’s  time  to  rewrite  the  rules. 

COLD  WAR  RESPONSES 

Eckes  showed  that  a  substantial 
portion  of  the  loss  of  the  industrial  base  has 
been  due  to  a  deliberate  sacrifice  of  Ameri¬ 
can  international  trade  to  political  consider¬ 
ations  attendant  on  the  Cold  War.  "For  45 
years  a  succession  of  presidents,  beginning 
with  Harry  Truman,  have  consciously  subor¬ 
dinated  domestic  economic  interests  to 
foreign  policy  objectives.  To  strengthen  free 
world  economies  and  help  contain  Soviet 
expansionism  the  executive  branch  has  rolled 
back  tariffs  and  removed  trade  restrictions, 
opening  the  giant  American  market  to  the 


world’s  manufacturers  [4]." 

The  Cold  War  is  over.  These  trade¬ 
offs  are  no  longer  necessary.  Responses 
formerly  not  considered  reasonable  are  now 
feasible. 


NEW  RESPONSES 

The  shrinkage  of  the  defense  portion 
of  the  U.S.  industrial  economy  has  been  part 
of  the  contraction  of  the  overall  industrial 
base.  This  is  now  being  compounded  by  the 
drawdown  of  the  military  services  and  atten¬ 
dant  additional  loss  of  military  contracts  due 
to  the  end  of  the  Cold  War. 

Resolving  issues  of  the  overall  econo¬ 
my  will  go  a  long  way  towards  restoring  the 
defense  industrial  base. 

The  loss  of  significant  portions  of 
steel,  a'-*os,  machine  tools,  shoes,  clothing, 
telecomiA  anications,  computers,  computer 
chips,  copying  machines,  televisions,  VCRs, 
toys,  etc.,  to  foreign  suppliers  has  weakened 
the  overall  economy  and  defense. 

Solutions  can  be  broken  into  two 
parts,  the  overall  economy  and  the  primary 
defense  suppliers.  By  overall  economy  we 
mean  steel,  autos,  machine  tools,  shoes, 
clothing,  telecommunications,  computers, 
computer  chips,  copying  machines,  televi¬ 
sions,  VCRs,  toys,  etc.,  products  which  are 
useful  in  both  the  civilian  and  military  sec¬ 
tors.  The  overall  economy  is  a  support 
structure  for  the  defense  industrial  base. 

Primary  defense  suppliers  are  con¬ 
tractors  directly  making  weapons  systems, 
such  as  General  Dynamics,  McDonnell 
Douglas,  General  Electric,  Pratt  and  Whit¬ 
ney,  etc.,  manufacturers  and  assemblers  of 
guns,  missiles,  tanks,  airplanes,  and  ships. 
In  addition,  manufacturers  whose  products 
are  direct  components  of  weapons  systems 
may  be  considered  primary  defense  manu¬ 
facturers. 
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We  first  deal  with  the  overall  econo¬ 
my. 

RECIPROCITY 

The  first  consideration  is  reciprocity. 
The  United  States  must  demand  and  obtain 
full  reciprocity  in  all  trade  matters.  The 
Cold  War  is  over.  There  is  no  longer  a  need 
to  sacrifice  our  economy  to  maintain  the 
support  of  our  allies.  Any  rights  and  privi¬ 
leges  already  granted  foreign  nations  and 
companies,  without  full  reciprocity  for  the 
United  States,  here  and  abroad,  should  be 
withdrawn. 

If  American  companies  cannot  buy 
into  foreign  companies,  such  rights  should 
be  withdrawn  from  foreign  companies. 
Where  granting  reciprocal  rights,  because  of 
ihe  relative  size  and  quality  of  the  markets, 
would  lead  to  an  uneven  exchange,  to  the 
disadvantage  of  the  U.S.,  such  reciprocal 
rights  would  not  be  granted. 

Recently  a  major  defense  newspaper 
advocated  that  foreign  interests  be  permitted 
to  buy  into  American  defense  contractors 
[13].  Since  such  rights  are  not  granted  U.S. 
companies  elsewhere,  such  buy-ins  should 
not  be  permitted  here. 

TARIFFS  AND  THE  VCT 

Tariffs  have  been  anathema  for  the 
last  60  years,  particularly  since  World  War 
II.  This  has  been  a  legacy  of  Adam  Smith 
and  "free  trade."  However  they  must  be 
considered  in  a  less  restrictive  manner. 

Discussions  of  trade  issues  have  been 
poisoned  by  one  word,  "protectionism."  It 
has  been  confused  with  another  word,  "mer¬ 
cantilism."  Mercantilism  is  the  practice  of 
attempting  to  build  the  largest  international 
trade  surplus  possible.  In  Smith’s  time  it 
was  measured  in  gold  accumulated  in  the  na¬ 
tional  treasury.  He  rightly  condemned  it. 

Tariffs  were  an  important  part  of 


mercantilist  policy.  Thus  tariffs  have  his¬ 
torically  been  condemned  as  part  of  mercan¬ 
tilism.  The  use  of  tariffs  to  "protect"  domes¬ 
tic  industry  began  to  be  called  "protection¬ 
ism."  Protectionism  became  a  dirty  word, 
used  in  place  of  mercantilism. 

Paradoxically,  mercantilism  is  now 
respectable.  It  masquerades  under  the  name 
"export  drive."  Japan,  South  Korea,  Taiwan, 
Hong  Kong,  Singapore,  Brazil,  China  and 
other  major  trading  nations  are  all  practi¬ 
tioners,  meanwhile  professing  free  trade. 
The  purpose  of  modern  mercantilism  is  less 
the  hoarding  of  gold,  than  the  hoarding  of 
industry  and  jobs.  We  might  call  it  "job 
mercantilism." 

If  a  nation  is  inundated  by  imports  as 
a  result  of  foreign  export  drives,  its  own 
industries  threatened,  and  it  responds  with 
tariffs,  it  is  condemned  as  protectionist.  It 
has  become  wrong  to  protect  yourself  if 
attacked. 

Tariffs  need  not  be  part  of  a  mercan¬ 
tilist  policy.  They  can  be  used  to  defend 
against  the  mercantilism  of  others.  They 
shouldn’t  be  condemned  if  used  in  this  man¬ 
ner.  A  nation  with  a  large  trade  surplus 
which  maintained  high  tariffs  might  be  con¬ 
demned  as  mercantilist.  One  with  substan¬ 
tial  trade  deficits  is  justified  in  using  tariffs. 
The  alternatives  are  quotas,  voluntary  export 
restraints  (VERs)  and  other  gimmicks.  Even 
free  trade  proponents  consider  tariffs  to  be 
superior  to  these  contrivances. 

Tariffs  are  a  tool.  They  can  be  used 
as  part  of  an  export  drive  or  they  can  be 
used  to  defend  against  the  export  drives  of 
others.  In  the  first  case  they  are  a  mercan¬ 
tilist  device  and  rightfully  condemned.  In 
the  second  case  they  are  justified. 

The  air  would  be  cleared  if  "protec¬ 
tionism"  were  dropped  from  the  vocabulary 
of  economics.  If  mercantilism  is  meant,  it 
should  be  used.  Tariffs  should  be  accepted 
for  what  they  are,  a  tool  which  may  be  well 
used  or  ill  used. 
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THE  VARIABLE  COMPENSATORY 
TAX 

Free  trade  doesn’t  work.  Mercantil¬ 
ism  and  export  drives  are  to  be  condemned. 
An  industrial  policy  has  little  chance  of  suc¬ 
cess  in  the  United  States.  What  should  be 
done? 

A  Variable  Compensatory  Tax  (VCT) 
has  been  proposed  [3].  It  is  a  system  of 
self-adjusting  tariffs  used  to  balance  trade.  It 
would  bring  U.S.  trade  into  a  relative  bal¬ 
ance  as  follows: 

-.05  <(E-I)/E  <  +.05 
where  E  is  exports  and  I  is  imports. 

To  initialize  the  system,  first  all 
existing  tariffs  and  other  trade  barriers  ex¬ 
cept  health,  environmental  protection,  and 
safety  would  be  removed.  Then  an  across- 
the-board  tariff  of  10  percent  would  be  ap¬ 
plied  to  products  of  nations  with  which  the 
U.S.  has  been  more  than  5  percent  in  cumu¬ 
lative  average  deficit  over  the  last  five  years. 
This  would  avoid  a  new  flood  of  imports  at¬ 
tracted  by  the  sudden  removal  of  other  trade 
barriers.  Figure  2  illustrates. 

Each  year  the  U.S.  cumulative  aver¬ 
age  five  year  trade  balance  would  be  re¬ 
viewed.  If  more  than  5  percent  in  deficit, 
the  tariff,  against  all  products  of  nations  with 
which  the  U.S.  is  more  than  5  percent  in 
deficit,  would  be  raised  by  5  percent.  If  5 
percent  or  more  in  surplus,  the  VCT  would 
be  reduced  by  2  percent  against  all  products 
of  nations  with  which  the  U.S.  is  5  percent 
or  more  in  surplus. 

No  action  would  be  taken  towards 
nations  within  the  +5  percent  limits.  Five 
year  cumulative  average  balances  are  used  as 
the  annual  measure  of  the  U.S.  balance  to 
stabilize  the  system  and  even  out  peaks. 

There  are  other  options,  but  they  all 
require  picking  and  choosing  industries  to  be 
protected,  and  selecting  a  method  and  level 
of  protection.  These  present  very  difficult 
issues  and  open  the  way  for  political  rather 


than  economic  solutions. 

WHAT  WOULD  THE  VCT  DO? 

The  objective  of  the  VCT  is  to  bring 
U.S.  trade  into,  and  maintain  it  within,  a  bal¬ 
ance  of  +  5  percent.  It  is  simple  and  self- 
enforcing.  There  would  be  at  most  150 
tariffs,  one  for  each  nation.  No  new  bureau¬ 
cracies  are  needed  to  implement  it.  In  fact 
the  Department  of  Commerce  could  be 
shrunk  by  several  thousand  persons,  now 
used  to  manage  our  chaotic  trade  laws. 

The  VCT  is  a  cybernetic  process, 
thus  self-correcting.  The  more  foreign 
nations  bought  here,  the  more  they  could 
sell;  the  less  they  bought,  the  less  they  could 
sell.  The  VCT  would  automatically  control 
quantities  by  adjusting  the  tariff  up  or  down, 
according  to  changes  in  the  trade  balance. 
Thus  any  attempt  at  retaliation  by  our  trad¬ 
ing  partners  would  be  self-defeating. 

As  a  side  effect,  the  VCT  would  bring 
back  the  industry  and  matching  6  million 
jobs  which  have  been  exported  in  the  last 
20  years. 

In  addition,  it  has  the  potential  to 
yield  up  to  $70  billion  in  revenue,  helping 
reduce  the  federal  deficit.  During  the  19th 
century  the  tariff  was  the  chief  source  of 
federal  revenue.  Far  from  harming  the  U.S. , 
during  that  period  it  built  the  mighty  indus¬ 
trial  machine  now  under  severe  pressure. 

WOULD  AMERICAN  PRICES  GO  UP? 

Not  necessarily.  The  price  of  goods 
depends  on  many  factors  including  profits, 
the  costs  of  labor,  raw  materials,  transporta¬ 
tion,  taxes,  and  overheads.  Many  of  these 
are  volume  dependent.  If  there  is  sufficient 
volume  more  mechanization  is  justified. 
This  reduces  the  unit  cost  of  labor. 

In  addition  another  phenomenon 
takes  place,  learning.  As  managers  and 
workers  learn  their  jobs,  they  become  better 
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Figure  2 


Trade  Management  Diagram 


Dollar  Imports 


at  them.  Production  lines  are  shortened. 
More  mechanization  and  automation  is  intro¬ 
duced.  The  company  learns  how  to  use 
clieaper  raw  materials.  As  volume  increases 
the  industry  comes  down  the  learning  curve. 
Costs  and  prices  also  come  down. 

When  huge  volumes  of  steel,  autos, 
machine  tools,  shoes,  clothing,  telecommuni¬ 
cations,  computers,  computer  chips,  copying 
machines,  televisions,  VCRs,  toys,  etc.,  and 
other  home  and  business  electronic  industries 
are  repatriated,  foreign  companies  will  lose 
significant  production  volumes.  American 
companies  will  gain  them.  As  American 
costs  and  prices  come  down,  foreign  ones 
will  go  up.  American  manufacturers  will 
become  more  competitive  both  at  home  and 
abroad. 

Even  if  some  American  prices  are  not 
fully  competitive,  our  international  trade  will 
not  suffer.  If  U.S.  steel  prices  are  higher 
than  they  otherwise  might  be,  affecting 
American  car  prices,  U.S.  car  exports  may 
be  reduced.  However  if  foreigners  did  not 
buy  American  cars,  they  would  have  to  buy 
computers,  TVs  or  other  products  if  they 
wished  to  export  to  the  United  States.  The 
VCT  would  automatically  compensate  for 
imbalances. 

A  net  20  percent  of  goods  consumed 
in  the  United  States  are  now  imported.  If 
employment  had  grown  along  with  the 
population  in  the  last  20  years,  there  would 
be  about  6  million  more  persons  employed 
in  manufacturing.  Since  there  has  been  no 
growth  in  manufacturing  employment,  we 
may  conclude  that  the  goods  to  supply  the 
50  million  person  population  growth  have 
come  at  the  expense  of  6  million  excess 
Americans  without  jobs.  When  trade  came 
into  balance,  the  VCT  would  will  have 
brought  back  about  6  million  jobs  and  the 
attendant  industries,  representing  an  in¬ 
crease  of  20  percent  in  U.S.  manufacturing 
capacity. 


THE  DEFENSE  INDUSTRIAL  BASE 

Our  armed  forces  have  the  responsi¬ 
bility  to  be  prepared  for  all  manner  of  con¬ 
flicts,  from  major  wars  to  skirmishes.  This 
can  only  be  accomplished  if  the  defense  in¬ 
dustrial  base  is  in  a  position  to  support  the 
conflicts  which  may  arise. 

Saddam  Hussein  and  Iraq  are  still 
dangerous.  No  one  knows  when  future  ty¬ 
rants,  even  more  dangerous,  will  arise.  The 
former  Soviet  Union  and  China  remain 
potentially  dangerous. 

Repatriation  of  20  percent  of  U.S. 
industrial  capacity  will  provide  the  entire 
economy  a  large  boost,  including  all  support¬ 
ing  structures  for  the  defense  industrial  base. 

It  will  do  little  for  direct  defense 
manufacturers  of  guns,  missiles,  ships,  tanks, 
and  airplanes  and  many  of  their  immediate 
components. 

This  issue  is  more  difficult.  Military 
equipment  needs  are  significantly  less,  in  an 
era  without  a  major  United  States  enemy,  yet 
it  is  important  that  the  manufacturing  capa¬ 
bility  be  maintained. 

The  Congressional  Budget  Office  re¬ 
ports  that  the  three  industries  most  depen¬ 
dent  upon  military  purchases  are  tank  and 
tank  components  (100  percent),  shipbuilding 
and  repair  (99  percent),  and  complete  guided 
missiles  (84  percent)  [1].  Others  with  very 
high  percentages  are  artillery  and  other 
heavy  weapons,  along  with  their  ammuni¬ 
tion. 

There  are  two  industrial  sectors 
which  can  be  directly  supported  through 
economic  policy,  shipbuilding  and  aircraft. 
Each  have  major  civilian  components. 
Sectors  such  as  tanks,  guns  and  missiles  are 
more  difficult  to  deal  with. 

SHIPBUILDING  AND  SHIPPING 

The  state  of  the  U.S.  shipping  indus¬ 
try  is  measured  by  the  following  figures: 
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since  1947  the  numbers  of  active  U.S.  flag 
ships  has  dropped  from  about  2,114  to  369 
and  the  number  of  billets  from  100,000  to 
11,000.  Meanwhile  new  construction  has 
dropped  to  near  zero  [6,11]. 

The  repatriation  of  20  percent  of  the 
American  manufacturing  economy  will  not 
assure  that  any  one  sector  has  any  particular 
level  of  operation.  It  is  entirely  possible  that 
American  international  shipbuilding  and 
shipping  will  continue  to  be  almost  non-exis¬ 
tent.  If  after  a  five  year  period  of  operation 
of  the  VCT  there  is  no  major  recovery  in 
U.S.  shipbuilding  and  international  shipping, 
more  direct  measures  are  called  for. 

The  simplest  solution  is  a  decision 
that  50  percent  of  U.S.  international  trade 
will  go  in  U.S.  bottoms.  This  means  50  per¬ 
cent  U.S.  built,  U.S.  owned,  U.S.  flagged, 
and  100  percent  manned  by  U.S.  citizens. 

This  could  be  accomplished  by  im¬ 
plementation  of  a  port  tax  [2].  It  would 
work  as  follows:  All  vessels  would  be  clas¬ 
sified  as  petroleum  tankers,  other  tanker, 
cruise/ passenger,  general  merchandise,  roll- 
on/roll-off,  or  other  classification  which  may 
be  deemed  reasonable. 

Each  vessel,  excluding  U.S.  intra¬ 
coastal  trade,  would  be  identified  by  class 
and  size  (deadweight  tonnage).  The  port 
entries  of  foreign  and  U.S.  vessels  would  be 
logged.  To  initialize  the  system,  a  $15  per 
100  vessel  deadweight  tons,  per  port  entry, 
tax  would  be  levied  on  foreign  vessels.  This 
tax  would  be  raised  by  10  percent  annually 
until  the  American  portion  of  international 
shipping  in  American  ports  reached  45  per¬ 
cent  of  the  tonnage.  Existing  shipping 
subsidies  would  be  gradually  phased  out. 

The  tonnage  would  be  reviewed 
annually.  If  the  American  portion  was 
within  a  limit  of  50  percent,  +  5  percent,  no 
changes  would  be  made.  If  under  45  percent 
American,  the  port  tax  would  be  adjusted 
upwards  by  5  percent.  Conversely,  if  over 
55  percent  American,  the  port  tax  would  be 


reduced  by  5  percent. 

During  the  first  eight  years,  foreign 
built  ships  would  be  excluded  from  the  port 
tax,  if  they  meet  all  other  qualifications 
(U.S.  ownership,  etc).  This  would  provide  a 
period  during  which  the  newly  revitalized 
U.S.  shipbuilding  industry  could  ramp  up  to 
the  necessary  production  level.  During  the 
eight  year  initiation  period,  any  federal 
shipping  subsidies  would  be  gradually  elimi¬ 
nated. 

In  this  manner  the  cost  of  shipping  in 
foreign  vessels  would  be  raised  high  enough 
so  that  U.S.  merchantmen  would  once  again 
become  competitive.  The  U.S.  share  of  in¬ 
ternational  ocean  transport  would  vary  from 
45  to  55  percent. 

As  U.S.  shipping  came  into  balance 
shipyards  would  be  revitalized  and  the  entire 
U.S.  merchant  marine  would  gain  a  new 
lease  on  life. 

Shipyards  would  no  longer  be  almost 
100  percent  dependent  on  the  U.S.  Navy  for 
their  livelihood.  If  a  major  new  naval  con¬ 
struction  program  were  required  the  facilities 
would  be  in  place. 

The  President’s  Commission  pro¬ 
posed  subsidies  and  a  host  of  catch-as-catch- 
can  measures.  These  would  end  up  costing 
the  taxpayer  money.  A  port  tax  would 
generate  revenue  as  it  brought  U.S.  shipping 
into  balance. 

THE  AIRCRAFT  INDUSTRY 

Issues  in  the  aircraft  industry  are 
somewhat  different.  Fortunately  this  indus¬ 
try  is  relatively  strong  compared  to  the  co¬ 
matose  shipbuilding  industry.  However  it  is 
under  attack  on  several  fronts. 

Foreign  airlines  have  been  and  are 
buying  into  American  airlines.  Recently 
KLM  bought  49  percent  of  Northwest  Air¬ 
lines.  Air  Canada  is  buying  into  Continental 
and  British  Airways  tried  unsuccessfully  to 
buy  44  percent  of  USAir  stock  [9]. 
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This  has  been  permitted  despite  the 
lack  of  reciprocity  [7].  An  American  airline 
would  not  even  consider  buying  into  a  for¬ 
eign  one.  It  would  be  a  meaningless  ges¬ 
ture,  an  assault  on  the  pride  of  the  foreign 
governments  and  their  people. 

This  buy-in  to  American  airlines  has 
ominous  overtones.  Though  technically 
remaining  in  American  control,  foreign  inter¬ 
ests  would  ultimately  be  in  a  position  to 
influence  airline  purchases.  This  is  good 
news  for  Airbus,  bad  news  for  Boeing  and 
McDonnell  Douglas. 

Not  only  would  the  civilian  compo¬ 
nent  be  damaged,  but  their,  and  all  then- 
suppliers’,  abilities  to  support  a  major  Amer¬ 
ican  airpower  build-up  would  be  diminished. 

Interestingly  a  major  defense  newspa¬ 
per  advocated  permitting  foreign  companies 
to  buy  into  defense  contractors  [13]. 

GUNS,  TANKS,  FIGHTERS 

Guns,  tanks,  fighters  and  similar 
equipment  are  essentially  specialized  to  the 
military  services.  There  are  no  civilian  uses 
for  155  mm  cannon.  Ml  tanks  or  F-16 
fighters. 

Production  decisions  with  respect  to 
these  types  of  equipment  in  an  era  of  very 
limited  demand  leave  two  choices,  low  rates 
of  production  and/or  intermittent  production. 

If  low  rates  of  production  is  the 
selected  alternative,  assembly  lines  would 
kept  open  for  longer  periods  of  time.  A 
trained  workforce  would  be  maintained 
which  can  act  as  a  cadre,  if  it  is  necessary  to 
ramp  up. 

If  required  to  make  engineering 
changes  or  upgrades,  based  on  experience 
with  early  production  models,  there  would 
be  fewer  items  needing  such  upgrades. 
Since  production  is  at  a  low  rate,  relatively 
few  would  have  been  produced  when  the 
new  needs  become  known. 


If  higher  rates  of  production  are 
selected,  unit  costs  may  be  lower  but  the 
assembly  lines  will  be  shut  down  for  longer 
periods  between  production  runs,  carrying  a 
risk  if  production  is  suddenly  needed  during 
a  period  of  shutdown. 

Each  case  must  be  separately  re¬ 
viewed,  tradc-ofts  conducted,  and  decisions 
made  on  ?ii  individual  basis. 

FMS,  COPRODUCTION,  TECHNOLOGY 

Foreign  military  sales  (FMS)  are 
desirable  from  a  cost  point  of  view.  They 
enable  factories  to  maintain  longer  produc¬ 
tion  runs,  helping  reduce  unit  costs.  They 
also  keep  assembly  lines  open  longer  than 
otherwise.  The  risk  is  that  the  customer  may 
some  day  turn  on  us,  causing  us  to  have  to 
fight  our  own  equipment. 

Co-production  is  not  as  attractive  as 
it  may  seem.  If  part  of  the  component 
production  is  farmed  out  overseas,  U.S. 
industry  will  lose  the  associated  component 
production  volume,  negating  the  effect  of  the 
increased  weapons  systems  volume.  In  addi¬ 
tion  it  often  entails  technology  transfer. 

Technology  transfer  has  two  risks, 
first  that  the  technology  will  be  leaked  to 
unfriendly  nations,  as  in  the  Toshiba/Kongs- 
berg  affair.  Secondly,  it  strengthens  foreign 
commeicial  competitors.  The  commercial 
competitors  are  in  competition  with  our  own 
industry.  American  technology,  paid  for  by 
taxpayer  dollars,  becomes  a  source  of  com¬ 
petition  for  American  industry. 

If  either  happens,  it  is  equivalent  to 
chasing  our  own  tail,  hardly  a  winning 
proposition. 

CONCLUSIONS 

Saving  the  defense  industrial  base  is 
a  doable  proposition.  Two  sets  of  interac¬ 
tions  exist;  the  relationship  of  the  world  eco- 
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nomy  to  the  civilian  industrial  base  and  that 
of  the  civilian  economy  to  the  defense  sec¬ 
tor. 

Despite  the  comment  of  a  high  level 
economic  advisor  to  the  president,  quoted  in 
the  introduction,  potato  chips  are  not  eco¬ 
nomically  equivalent  to  computer  chips  in 
the  real  world.  The  comment  reflects  a 
cavalier  attitude  towards  a  precious  com¬ 
modity,  our  industrial  base. 

It  has  been  shown  that  every  presi¬ 
dent  from  Truman  to  Bush  has  deliberately 
sacrificed  American  economic  interests  to 
political  interests  in  fighting  the  Cold  War. 
The  damage  to  the  civilian  economy  has  in 
turn  seriously  weakened  the  defense  sector. 
We  have  become  dependent  on  imported 
electronics  and  other  goods  for  critical  de¬ 
fense  needs. 

The  Cold  War  is  over.  Approxi¬ 
mately  20  percent  net  of  all  American  goods 
are  imported.  They  represent  the  industrial 
capacity  of  steel  mills,  shipbuilding,  cloth¬ 
ing,  shoes,  computer  chips,  TV,  VCR  and 
other  high  and  low  tech  industry. 

Some  of  these  industries  no  longer 
exist  in  the  United  States.  Others  are  in  a 
greatly  reduced  state,  weakening  our  overall 
capacity  to  respond  to  defense  needs  in  a 
major  conflict. 

The  factories  are  gone,  the  trained 
people  are  gone. 

Responses  which  were  not  politically 
appropriate  are  now  acceptable.  By  bringing 
U.S.  trade  into  balance  with  the  VCT,  our 
industry  could  grow  by  20  percent  providing 
a  substantial  boost  to  our  overall  ability  to 
respond  to  a  defense  emergency. 

In  addition,  policies  dealing  with 
specific  defen.se  needs  must  be  addressed. 
Given  the  desire,  U.S.  shipbuilding  and 
shipping  can  be  returned  to  a  commercial 
success.  Our  aviation  industry  must  be  care¬ 
fully  nurtured,  lest  it  go  the  way  of  computer 
related  hi-tech,  or  worse  yet,  shipbuilding. 

When  dealing  with  all  i.ssues,  as 


much  attention  must  be  given  to  secondary 
effects  as  to  the  primary.  Sale  of  American 
airlines  to  foreign  interests  places  the  aircraft 
industry  at  risk.  Co-production  and  technol¬ 
ogy  transfer  weaken  U.S.  industry.  Amer¬ 
ican  tax  dollars  in  effect  fund  foreign  manu¬ 
facturers.  We  end  up  chasing  our  own  tails. 

These  issues  are  resolvable.  Specific 
policies  can  help  rebuild  the  industrial  base, 
giving  the  support  structure  for  the  defense 
industry  a  significant  boost. 

Issues  dealing  with  specialized  mili¬ 
tary  equipment  such  as  guns,  missiles,  tanks, 
and  fighter  planes  are  more  difficult.  Prob¬ 
lems  of  small  production  runs  cannot  be 
avoided.  However  extended  low  production 
rates,  FMS,  and  if  necessary  mothballing 
production  facilities  must  all  be  considered 
in  the  final  trade-offs. 
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CAN  WE  SIMPLIFY  HOW  WE  MEASURE  SUPPLIER  PERFORMANCE? 
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ABSTRACT 


By  combining  creativity  and 
research,  results  indicate  that 
one  can  simplify  the  method  by 
which  suppliers  and  sub¬ 
contractors  are  measured.  This 
paper  describes  several  examples 
of  measuring  supplier 
performance.  The  message  is 
three  fold. 


First,  a  performance 
criteria,  that  rewards 
productivity,  teamwork,  and 
entrepreneurship,  can  be 
standardized  for  both  customers 
and  suppliers.  The  difference  is 
in  how  it  is  measured.  Second, 
offering  something  free  that  is 
of  value  to  the  customer  serves 
as  a  first  indicator  of  service. 
Third,  the  second  indicator  is 
productivity  -  in  terms  of 
knowing  the  processes  or  steps, 
and  in  terms  of  specific 
repeatable  techniques,  is 
neither  well  understood,  nor 
practiced,  nor  measured  nor 
rewarded . 


The  examples  range  from  a 
tire  company,  a  training 
company,  auto  repair  service, 
insurance  company,  leasing 
company,  roofing  company,  to 
health  care  providers. 

In  all  instances, 
simplicity  and  specific 
productivity  techniques  in  terms 
of  repeatable  techniques  or 
processes,  were  observed.  The 
techniques  and  processes  are 
described,  along  with  measures 

and  rewards.  The  risks  in 
measuring  supplier  performance 
is  that  the  wrong  things  are 
often  measured. 


Seldom  is  productivity  in 
terms  of  processes  or  techniques 
measured  or  rewarded.  The 
solution  is  (1)  to  follow  a 
standard  performance  criteria 
and  measure  productivity  in 
terms  of  simplicity,  processes, 
and  repeatable  techniques,  for 
both  customers  and  suppliers, 
management  and  staff,  and  (2) 
measure  the  right  performance. 
The  performance  that  is  measured 
will  be  rewarded. 


INTRODUCTION 

Much  has  been  studied  and 
written  about  performance 
measures,  quality,  and 
acquisition.  But  few  understand 
what  to  look  for  and  measure  in 
determining  performance  in 
people  and  companies  who  offer 
service  and  products.  How  can 
you  measure  the  performance  of 
suppliers?  What  are  the  measures 
that  determine  quality 
performance? 

By  modifying  a  standard 
performance  criteria  (Figure  1) 
that  rewards  (1)  productivity 
(2)  entrepreneurship,  and  (3) 
teamwork,  and  applying  this  to 
various  supplier-based  research 
applications,  similarities  have 
been  observed. 

In  seeking  answers  for  "how 
to's"  in  the  war  against  waste, 
inefficiency  and  bureaucracy, 
one  has  to  examine:  How  are  we 
measuring  performance  now?  What 
SHOULD  be  measured? 

The  KEY  to  determining 
performance  is  (1)  the 
suppliers'  helpfulness  in 
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meeting  the  customer's  needs, 
and  (2)  knowledge  and  practice 
of  productivity  skills  _an_d 
techniques ♦  People,  suppliers  of 
all  kinds,  should  be  measured 
and  rewarded  for  using 
productivity  techniques  and 
processes  in  some  way.  Realizing 
that  rewards  and  measures  are 
often  intricately  intertwined  is 
key  to  understanding  and 
measuring  performance  correctly. 

The  most  difficult  part  of 
rewarding  the  right  behavior  is 
determining  the  right  things  to 
measure.  Frequently  the  wrong 
activities  are  measured  and  the 
right  ones  are  ignored. 

There  are  two  steps  to 
measuring  performance;  (1)  offer 
something  free  that  is  of  value 
to  the  customer  and  (2)  seek 
suppliers  who  use  some 
productivity  techniques  that 
offer  savings  of  effort  and 
money  to  the  customer.  Let's 
examine  them  briefly. 


Offer  something  of  value 
free  to  the  customer;  The  key 
here  is  understanding  what  the 
customer  needs  in  order  for  the 
offering  to  be  of  value  to  them. 
It’s  the  little  things  that 
count . 

People  seeking  service 
and/or  products  of  any  kind- 
whether  it  be  roofers, 
consultants,  lawyers,  health 
care  providers  -  should  seek  and 
listen  for  ways  the  supplier  can 
save  them  money,  time,  or 
effort.  An  effective  supplier 
should  inform  and  educate  the 
buyer  by  providing  simple, 
helpful,  valuable  information  - 
provided  freely-  which  assists 
the  buyer  in  being  a  more 
informed  purchaser  and  meeting 
his  needs,  be  they  large  or 
small.  Something  offered  freely 
must  be  of  value  to  the 
customer.  If  it  is  offered 
freely  but  is  not  of  value  to 
the  customer,  the  free  offer 
will  not  be  effective.  Getting 
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to  know  the  needs  of  the 
customer-  through  the  process  of 
listening  within  the  process  of 
negotiation  -  is  the  first  key 
to  identifying  quality  supplier 
performance . 

Services  of  value  to  the 
customer  can  only  occur  by 
listening  to  the  customer  and 
determining  what  might  be 
helpful  to  meet  his  needs. 
Effective  listening  to  ''where 
the  person  is  coming  from"  and 
"needs"  would  help  the  supplier 
provide  small  yet  extremely 
important  free  services,  ...key 
to  establishing  oneself  as  a 
quality  supplier. 

Example;  Melissa,  an  11 
year  old,  went  to  a  tire  company 
v;ith  her  four  quarters  to  buy  an 
inner  tube  for  sledding.  When 
the  tireman  heard  her  request, 
he  found  a  rumpled  tube,  cleaned 
it  up,  filled  it  with  air,  and 
gave  it  to  her  freely. 

Comment :  This  example  of 
something  offered  freely  to  meet 
the  customer's  needs  is  the 
first  indication  of  service  and 
good  will. 

Example:  A  training  company 
offered  free  registration  to 
this  author  to  attend  an  out-of- 
town  class  on  planning.  Since 
the  customer  (this  author) 
teaches  planning,  this  free 
offer  was  of  little  value.  A 
textbook  was  offered  as  part  of 
the  course  and  the  customer 
would  have  liked  a  free  copy  of 
the  text.  An  earlier  request  for 
a  free  review  copy  was  denied. 

Comment ;  Had  the  course 
provider  agreed  to  send  a  free 
copy  of  the  text,  the  "free 
book"  gesture  would  have  been 
less  expensive  in  dollars  for 
the  supplier  and  more  valuable 


to  the  customer.  It  is  the 
responsibility  of  the  supplier 
to  listen  to  the  needs  and 
requests  of  the  customer  and 
respond  to  those  needs,  or 
attempt  the  negotiating  process 
by  offering  alternatives. 
Frequently  customer  requests  and 
solutions  are  more  simple  than 
suppliers  might  imagine. 


COMBINING  FREE  WITH  SIMPLICITY 

Occasionally,  something 
offered  freely  is  combined  with 
simple  solutions.  Consider  the 
following. . . 

Measure:  Simplicity.  Customer 
service.  Free. 

Reward:  Productivity. 

Example  -  Auto  Part  1;  They 
were  leaving  on  a  weekend  trip. 
The  driver  did  not  have  time  to 
check  out  the  car.  She  drove 
into  a  gas/repair  station  and 
asked  the  mechanic  if  he  could 
check  out  the  car  for  the  trip. 
He  explained  the  major  areas  to 
be  concerned  about,  checked  the 
fluids,  and  okayed  the  car  for 
the  trip,  in  about  5  minutes. 
Given  the  length  of  the  trip, 
the  mechanic  explained  that  this 
quick  check  was  adequate  to 
assure  the  driver  and  passengers 
that  the  auto  was  in  good 
condition  for  the  weekend  trip. 

Comment ;  This  quick  and 
efficient  assessment,  given  the 
parameters  provided  by  the 
customer,  provided  valuable  free 
information  for  the  customer's 
needs  at  hand. 

Example  -  Auto  Part  2:  The 
tire  was  very  low,  possibly  in 
need  of  replacing.  When  the 
mechanic  evaluated  the  tire,  he 
found  a  leak  in  the  rim/tire. 
The  mechanic  suggested:  (1) 
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replace  the  rim/tire  or  (2) 
insert  an  inner  tube  in  the 
tire.  The  dollar  savings  in  the 
simpler  choice  was  about  $40.00. 

Comment :  By  offering  the 
customer  choices,  one  of  which 
being  a  low  cost,  simple 
solution,  the  supplier  is 
illustrating  simplicity.  Often, 
quick  fix,  simple  solutions  will 
suit  the  needs  at  hand.  It  is 
important  for  the  supplier  to 
understand  the  requirements  and 
limitations  affecting  the 
customer  and  offer  sir.ple,  often 
creative  low  cost  solutions  and 
choices  that  work.  Is  it  really 
necessary  to  calculate  in 
dollars  and  minutes  the  savings? 
Isn't  the  goal  of  quality  to 
meet  the  customer's  needs?  If 
needs  are  met,  must  we  measure 
further? 

Measure;  Simplicity.  Something 
written.  Customer  service. 

Reward;  Productivity.  Teamwork. 
Creativity. 

Example;  A  fire  insurance 
policy  was  cancelled  on  a 
Florida  rental  property. 
Discussions  from  Seattle  with  an 
independent  agent  suggested  a 
complicated  and  expensive 
solution.  Another  agent 

suggested  a  simple  solution; 
"send  me  a  photo  of  the 
property,  confirm  that  it  has 
been  rented,  and  the  policy  can 
be  reinstated."  The  agent 
offered  to  contact  the  mortgage 
bank  and  coordinate  information 
and  efforts.  Simple. 
Cooperative. 


Comment ;  Seek  suppliers  who 
offer  streamlined,  simple 
solutions.  Seek  suppliers  who 
offer  to  do  something  extra  as  a 
way  of  offering  customer 
service,  perhaps  by  contacting 
others  to  make  it  easier  for 
you,  the  customer.  Look  for 
simplicity  and  teamwork  and 
reward  it.  The  agent  was 
rewarded  with  "praise"  tor  her 
assistance  and  simplicity  in 
solving  the  problem  via  a  "thank 
you"  call  from  the  customer. 


WHAT  IS  PRODUCTIVITY? 

What  is  productivity?  It  is 
neither  well  understood,  nor 
practiced,  or  measured,  or 
rewarded  consistently.  In 
addition  to  simplicity, 
productivity  includes; 

*  smart  work,  instead  of 
busy  work 

*  quality  work,  instead  of 
fast  work. 

*  Smart  work  instead  of 
busy  work;  In  these  complex  busy 
times,  we  can  no  longer  continue 
to  work  the  same  way  as  we  have 
in  the  past.  We  need  to  think 
creatively  before  we  act,  find 
new  methods  to  accomplish  work, 
and  find  new  uses  for  existing 
resources.  We  need  to  streamline 
efforts,  and  reward  people  who 
give  thought  to,  suggest,  and 
implement  ways  to  lessen  work, 
while  maintaining  high  quality 
by  eliminating  errors  and 
rework. 

What  is  meant  by  smart 
work?  Smart  work  includes 
productivity  techniques  which, 
although  simple,  are  frequently 
neither  practiced  nor  measured 
nor  rewarded.  Productivity 
techniques  (Figure  2)  such  as 
"have  more  than  one  use  for 
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Productivity  Techniques 
in  Mind  Map  Format 


something",  "put  it  in  writing", 
"do  the  easiest  first",  "keep  it 
simple",  and  "design  with  the 
most  detailed  in  mind"... are 
techniques  that  should  be 
measured  and  rewarded.  For 
example. 

When  doing  or  starting 
anything,  do  the  easiest  first. 

When  solving  problems, 
writing  documents,  developing  a 
briefing,  designing  anything,  do 
the  easiest  part  first. 

Design  with  the  most 
detailed  in  mind. 

After  designing  the  easiest 
first,  move  next  to  the  most 
detailed  section.  Once  the  most 
detailed  part  or  section  is 
designed,  all  the  other  parts 
can  be  modified  to  accommodate 
them. 

Have  more  than  one  use  for 
something. 

Can  a  letter  or  document  be 
used  by  more  than  one  audience? 
By  designing  with  more  than  one 
use  as  much  as  possible 
[documents,  letters,  buildings, 
software,  bottle  caps,  offices, 
bathrooms,  and  other  engineered 
efforts],  effort,  resources,  and 
dollars  could  be  saved. 


Consider  using  the  mind 
mapping  tool,  whenever  writing 
anything. 

Mind  mapping  and  other  idea 
generating  tools  (1)  can  provide 
a  quick  method  of  note  taking 
and  (2)  can  be  used  as  a 
productivity  tool  in  writing  to 
save  time  and  effort.  Mind 
mapping  can  be  used  when  solving 
problems,  when  gathering  ideas 
for  a  WBS,  when  developing  draft 
sows,  in  developing  briefings, 
when  writing  anything,  when 
being  creative  such  as  an  idea 
generating  tool,  perhaps  to 

break  the  whole  into  parts  or 
combining  parts  into  whole. 


APPLICATIONS  USING  PRODUCTIVITY 
TECHNIQUES 

The  following  are  examples 
of  applying  various  productivity 
techniques  by  various  suppliers. 

Measures:  Clarity.  Simplicity. 

Something  written. 

Reward:  Productivity. 

Creativity.  Teamwork. 

Example:  Using  the 

productivity  technique  of  "have 

more  than  one  use  for 

something",  Eaton  Financial 
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Corporation  uses  their  customer 
number  as  part  of  their  invoice 
number.  For  example,  customer 
number  is  #  0162667;  the  invoice 
number  is  #016266702093.  This 
cuts  down  on  errors  and  serves 
as  a  double  check  when  problems 
arise.  Their  contracts  are 
v;ritten  in  simple,  clear  English 
providing  an  environment  of 
understanding  and  clarity,  much 
appreciated  by  their  customers. 

Comment ;  Simplicity  is 
illustrated  in  the  clear  writing 
they  use  in  their  contractual 
documents  that  describe  in 
clear,  understandable  English 
the  legalistic  terms  and 
conditions.  Using  the  customer 
number  as  part  of  their  invoice 
number  is  also  creative  and 
entrepreneurial.  The  customer's 
needs  are  met  by  these  simple 
offerings  of  clear  writing  and 
understandable  terms. 

Rewards  to  Eaton  Financial 
Corporation  are  in  the  form  of 
"money"  (more  contracts)  and 
"praise"  as  people  use  Eaton  as 
a  benchmark  of  excellence  in 
modeling  simplicity  of  terms, 
contracts,  and  invoices. 


*  Quality  work,  instead  of 
fast  work,  means  understanding 

the _ processes.  Quality 

improvements  require  us  to  think 
in  terms  of  structured 
processes;  therefore  we  must 
seek  to  understand  them. 
Structured  methods  deal  with 
analysis,  design  and  development 
which  are  at  the  heart  of  the 
life  cycle.  Structured  methods 
or  systems  translate  functions 
into  a  set  of  steps.  Most 
everything  performed  has  some 
structure  to  it. 

Many  people  are 
implementing,  managing  and 
measuring  quality  improvements 
without  an  understanding  of  the 
processes,  methods  and  their 
integration.  Let's  examine  these 
processes  briefly. 

>  Life  Cycle  Process 
(Figure  3)  :  There  is  a  distinct 
difference  between  the  product 
to  be  developed  and  the  process 
that  builds  the  product.  The 
process  that  builds  the  product 
is  the  Life  Cycle  process  whose 
steps  can  be  simplified  to: 
Define,  Design,  Develop  and 
Deliver.  Sometimes  the  phases 
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are  understood  but  little 
attention  is  given  to  the  role 
of  the  customer,  people  skills 
and  processes,  and  the  value  of 
Preliminary,  Critical,  and 
System  Design  Reviews. 
Untangling  the  (1)  process  of 
building  the  product  (life 
cycle)  with  (2)  the  process  of 
planning  and  controlling  the 
work  (planning  and  controls 
process) . . .would  assist  managers 
in  feeling  more  secure  in 
understanding  how  to  structure 
and  manage  their  work  and  how 
this  work  fits  into  the  bigger 
picture. 

Sometimes  these  life  cycle 
processes  occur  very  rapidly  and 
so  simply  that  we  are  not  even 
aware  of  the  process. 

Measure;  Simplicity.  Reviews. 

Reward:  Productivity.  Teamwork. 
Creativity. 

Example :  She  had  a 
recurring  inflammation  on  and 
around  her  exterior  ear.  Various 
OTC  ointments  were  applied  with 
little  success.  Another 
pharmacist  listened,  and 
suggested  that  a  stronger  OTC 
ointment  be  used,  offering 
reasons  why  previous  therapy  may 
not  have  worked.  The  remedy  was 
tried;  the  problem  was  resolved 
almost  overnight. 

Comment :  The  pharmacist 
used  several  productivity 
techniques;  "keep  it  simple", 
"do  the  easiest  first",  and 
offered  free  valuable  advice. 
The  pharmacist  explained  why  the 
remedy  may  not  have  worked 
earlier  —  valuable  information, 
modified  the  course  of  action, 
and  recommended  a  simple,  easy 
solution  to  try.  From  the 
patient's  point  of  view,  this 
action  was  also  a  progress 


review,  in  the  life  cycle  of 
solving  this  recurring  ear 
problem,  where  the  problem 
solving  process,  plus  additional 
facts  from  the  new  team  member— 
supplier  (pharmacist) ,  led  to 
modifying  the  course  of  action, 
and  testing  the  results.  The 
pharmacist  suggested  a  "simple", 
"easiest  first"  work-around  plan 
solution  which  worked. 

>  Planning  and  Control 
Process  (Figure  4):  The  process 
of  planning  and  controls  is  an 
existing  and  effective  process 
used  within  aerospace  and 
defense.  There  are  specific 

steps  to  this  process.  It  is 
integrated  with  and  is  a  part  of 
the  life  cycle  process.  These 
steps-  SOW,  WBS,  schedules-  if 
structured  correctly,  can  be 
measures  of  quality  in  the 
planning  process.  Every  time 
work  has  to  begin  or  an 

assignment  is  given  to  anyone, 
the  planning  and  control  (or 
monitoring)  process  begins.  The 
process  already  exists  and  the 

steps  are  defined.  Perhaps  some 
tailoring  might  be  necessary  for 
a  specific  application. 

The  planning  and  control 
process  should  be  used  by 

managers,  self  empowered  teams, 
contractors ,  subcontractors 

...all  people  ...when  planning 
is  needed.  These  same  steps  can 
serve  both  as  performance 
measures,  and  communication 
tools  for  staff,  leaders, 
including  people  measuring  the 
performance  of  suppliers. 

Measure;  SOW,  WBS,  Schedules, 
and  other  documents  as  needed 
Reward;  Productivity.  Teamwork. 
Entrepreneurship . 

Example;  Dr.  Alan  R.  Carr, 
an  orthodontist,  after  the 
initial  free  introductory 
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meeting,  provides  the  patient 
with  a  big  picture  view  of  the 
orthodontic  life  cycle  approach 
planned  over  the  forthcoming 
years.  The  plan  is  pictorial, 
and  explained  in  understandable 
terms.  Then  he  develops  a  simple 
Statement  of  Work  for  his 
customers,  that  describes  in 
broad  terms  what  he  plans  to  do, 
major  milestone  dates,  payment 
terms  and  constraints,  and 
responsibilities . 

Signature  by  the  customer 
is  required  confirming  that  both 
customer  and  service  provider 
are  in  agreement.  The  documents 

and  procedures  he  uses  are  kept 
simple,  requesting  details  from 
the  patient  on  an  as-  needed 
basis  only,  eliminating 
unnecessary  duplication  of  data. 

Fun,  colored  rubber  bands, 
color  coding  of  the  charts  and 
other  administrative  records. 


use  of  recycled  stained  glass 
for  decoration. .. are  some  of  the 
many  examples  of  creativity  and 
entrepreneurship . 

His  staff  conducts  daily 
stand  up  review  meetings  to 
assure  that  all  staff  is  kept 
informed  of  specific  patient 
needs.  Information  is  shared. 
Communication  is  enhanced.  Staff 
is  kept  quickly  and  readily 
informed  of  problem  cases, 
current  issues,  and  daily 
happenings.  Staff  has  fun, 
shares  information,  works  each 
other's  functions,  and  is 
congenial  with  each  other  and 
the  customers. 

Progress  is  monitored  by 
monthly  review  meetings  (visits) 
with  the  customer  where  status 
is  discussed,  tests  are  made, 
modifications  occur  when  needed, 
developments  are  discussed,  and 
future  plans  are  shared. 


Comment ;  The  orthodontist 

(1)  offers  a  free  preliminary 
meeting  with  patients  to 
determine  if  he  can  provide  an 
appropriate  solution.  After 
agreeing  to  treatment,  he 
presents  his  treatment  (2)  using 
simple .  pictorial  documents  that 
describe  the  work  to  be 
performed  similar  to  a  statement 
of  work.  He  (3)  conducts 
periodic  reviews  and  tests 
examining  the  progress, 
providing  status  and  "next  step" 
plans  to  the  customer, 
throughout  the  life  cycle  of  the 
therapy.  (4)  If  problems  arise, 

the  orthodontist  modifies  and 
simplifies  his  approach  to  meet 
the  needs  of  the  customer,  a 
simple,  work-around  plan. 

This  supplier  offers  (1) 
quality  service  based  on 
valuable  information  offered 
freely  during  the  preliminary 
meeting,  (2)  simplicity  in 
streamlining  the  paperwork  and 
forms,  (3)  clarity  in  terms  of 
designing  and  developing  the 
course  of  therapy,  (4)  structure 


in  terms  of  statement  of  work- 
type  documents  in  planning,  and 
(5)  customer  service  and 
teamwork  seen  in  progress 
reviews  and  work-  around  plans. 

>  Writing  process  (Figure 
5)  :  Since  much  of  the  planning 
and  control  (managing)  function 
is  writing,  it  is  important  for 
leaders  to  understand  that 

(1)  writing  is  a  process 
which  has  structure, 

(2)  its  outcome  is  a 
document  (something  written) , 

( 3 )  the  document  must  be 
planned  and  managed,  (planning 
and  controls  structured  process) 
and 

(4)  the  document  has  a  life 
cycle  (Figure  6)  whose 
development  phases  must  be 
defined,  designed,  developed, 
and  delivered.  Note  that  the 
processes  have  logic  and 
structure,  repeat  themselves, 
and  are  continually  integrated. 

Measure;  Something  written. 
Various  productivity  techniques. 
Reward;  Productivity.  Teamwork. 
Entrepreneurship . 


The  Writing  Cycle 
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The  Document  Life  Cycle 
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User  Audit* 


Tost 


Test 


Definition 


Test  hte^ationTest 


Mi-ision/Assignment 
Statement  of  work 
Wcrfc  breakdown 
structure  (prelim) 

Audience 

Pu'pose 

Objectives 

Constraints 

Schedule 


Design 


Content  (prelim) 


Statement  of  Work  (Rev) 
Format  (prelim) 

Work  breakdown 
skucture  (Rev) 
Schedule  (Rev) 


Format  (Rev) 
Content  (Rev) 
Schedule  (Rev) 


Figure  6 


Format  {Frol  Rev) 
Content  (Fnal  Rev) 
Organization  (Final  Rev) 


F.xainple  ~  Auto  Part _ 3 1  The 

car  would  not  start,  appearing 
to  be  some  battery-related 
problem.  The  service  station 
mechanic  evaluated  the  problem, 
called  an  auto  electrical 
company,  and  conferred  with  the 

staff.  When  all  else  failed,  the 
service  station  staff  towed, 
with  no  tow  charges  to  the 
customer,  the  car  to/ from  the 
auto  electric  specialists,  who 
discovered  the  intermittent 
problem  was  with  the  alternator. 
The  auto  repair  dealer  chose  to 
replace  the  alternator  at  his 
location,  using  generic  parts, 
saving  the  customer  additional 
funds.  The  service  repair  owner 
delivered  the  repaired  car  to 
the  office  of  the  customer, 
adding  that  payment  could  be 
made  within  the  next  few  days. 


Comment ;  The  auto  mechanic 
-owner  practiced  (1)  ^  guality 

work  —  as  defined  in  this  paper 
under  productivity-  through  his 
trial  and  error,  modifications, 
and  successful  attempts  to  solve 
(problem  solving  process)  the 
electrical  problem  in  the  on¬ 
going  life  cycle  maintenance  of 
the  auto,  (2)  conducted  his  work 
around  plan  after  several  trial 
and  error,  test,  review,  and  try 
again  attempts  to  identify  the 
solution  in  the  life  cycle  of 
this  electric  problem,  (3)  used 
productivity  techniques  via 
generic  parts  ('*  have  more  than 
one  use  for  something")  ,  (4) 

practiced  teamwork  by  conferring 
with  electrical  specialist 
colleagues  and  by  providing 
feedback  to  the  customer,  (5) 
illustrated  trust  by  requesting 
payment  when  it  was  convenient 
for  the  customer,  and  (6) 
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produced  an  invoice,  written 
clearly,  that  identified  and 
described  the  charges. 

The  customer  was  rewarded 
with  quality  service.  The 
service  station  owner  was 
rewarded  with  "money"  (in  terms 
of  repeat  business)  and 
"praise".  Team  members  (ie, 
electrical  specialist  colleagues 

and  service  station  staff)  were 
rewarded  by  "praise". 


WRONG  MEASURES  OF  QUALITY 

Sometimes  friendliness  is 
mistaken  for  quality. 
Friendliness  by  itself,  while 
helpful,  is  not  a  measure  of 
quality.  Sometimes  other 
measures  are  mistaken  for 
quality  such  as  number  of  staff, 
spaciousness  of  the  office, 
degrees  and  licenses  earned,  or 
universities  attended.  These  are 
not  measures  of  quality. 

Example:  The  large,  well- 
known  roofing  repair  company  rep 
was  most  cordial  on  the 
telephone  and  provided  an 
estimate  of  the  repair  job  on 
the  rental  property.  The 
agreement  was:  pay  half  up 
front,  and  half  when  the  job  was 
finished.  Since  the  customer  was 
in  Seattle,  this  seemed 
appropriate.  When  the  customer 
received  the  bill  for  the  work 
performed,  it  seemed  to  be  a 
small  amount  of  work  for  the 
amount  charged.  In  seeking  a 
better  understanding  of  the  work 
involved  in  this  repair, 
although  the  repairs  occurred  in 
Florida,  some  roofers  in  Seattle 
were  called  and  the  repair 
situation  was  described.  One 
supplier,  when  questioned, 
provided  a  rough  estimate  of 
what  the  job  should  cost. 


Comment:  That  estimate 

free,  valuable  information  - 
assisted  the  purchaser,  located 
in  Seattle,  in  determining  that 
the  roofing  supplier,  located  in 
Florida,  was  overcharging  for 
the  job.  The  Florida  roofer  was 
informed  of  this  concern  and 
invited  to  justify  the  charges. 
Anger  was  the  reply  received. 
Friendliness  is  not  a  measure  of 

quality.  Charging  for  "service" 
is  also  not  a  measure  of 
quality.  Stating,  in  clear 
understandable  terms,  up  front, 
in  writing,  what  the  supplier 
includes  in  his  charges,  and 
what  is  excluded  is  a  measure  of 
quality  in  terms  of 

productivity,  simplicity,  and 
clarity  in  speaking  and 
writing. ...  clear ,  simple  terms 
to  understand,  offered  up  front, 
in  writing. 

Understanding  that  these 
processes  are  based  on  logic, 
repeatable,  and  standard,  is 
paramount  to  understanding  and 
recognizing  the  true  meaning  of 
quality  work.  Furthermore, 
understanding  (1)  that  most 
everything  has  a  process  and  (2) 
realizing,  in  simple  terms,  how 
these  processes  affect  us  in 

daily  life,  is  important  when 
seeking  to  recognize  these 
processes  in  suppliers. 


SUMMARY 

Using  a  standard 
performance  criteria  that 
measures  people  on  (1) 
productivity,  (2) 
entrepreneurship  and  (3) 
teamwork,  and  keeping  the 
measures  simple  and  focused  on 
productivity,  quality 
performance  in  suppliers  can  be 
observed,  recognized,  measured 
and  rewarded . 
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Too  often,  experts  in 
quality  seek  to  complicate 
measures.  When  deciding  which 
supplier  to  select,  ask 
yourself:  which  one  makes  work 
easier  and  simpler  for  me,  the 
customer?  Do  they  offer  simple, 
creative  suggestions,  and 
provide  streamlined  procedures? 
Do  they  offer  information  freely 
that  assists  you  in  being  a  more 
informed  purchaser?  Are  their 
documents,  contracts  and  terms 
easy  to  read?  Do  they  use 
pictures  instead  of  text  in 
describing  their  work?  Are  they 
clear?  Do  you  invite  and  reward 
creative  ideas  that  streamline 
procedures? 

Is  training  performance 
measured  by  the  size  of  the 
organization,  customers  who  have 
attended,  number  of  hours/people 
taught  (poor  measures)  or  number 
of  students  who  have  illustrated 
achievement  (good  measure)  in 
p^ -forming  the  functions? 

In  developing  software,  do 
your  contractors  measure  amount 
of  lines  of  code  written  and 
thereby  increase  run  time?  .... 
or  do  they  streamline  their 
coding,  write  it  in  a  structured 
fashion,  in  order  to  provide 
savings  in  maintena.;ce ,  in 
resources,  and  in  other  ways? 

The  measures  YOU  will  see 
will  be  ...(1)  information  or 
something  free  that  is  of  value 
to  you  in  achieving  your  goal  as 
a  customer,  and  (2)  streamlined, 
creative,  productive  methods 
that  save  time,  effort, 
resources  and  money.  In  addition 
to  observing  and  asking  about 
these  measures,  reward  suppliers 
for  streamlining,  simplicity, 
and  other  productive  measures 
and  you'll  benefit  from  their 
creativity  and  resourcefulness. 


If  performance  of  suppliers 
were  measured  and  depended  on 
seeking  and  initiating  creative, 
simplified  solutions,  and  in 
working  together  to  seek  and 
solve  customer  problems,  such  as 
the  examples  in  this  paper,  more 
performance  successes  would  be 
seen. 

Ask  yourself:  Are  you 
measuring  and  rewarding 
simplicity  and  productivity  in 
supplier  performance  now?  ## 


Reference: 
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ABSTRACT 

A  discounted  cash  flow  comparison  of 
acquisition  alternatives,  termed  "economic 
analysis"  by  the  Department  of  Defense, 
necessarily  focuses  on  the  quantifiable 
benefits  and  costs.  Nonquantifiable  factors 
are  much  more  difficult  to  include,  but 
should  not  be  ignored.  Saaty’s  Analytic 
Hierarchy  Process  (AHP)  is  one  way  to 
capture  both  quantifiable  and  nonquantifiable 
costs  and  benefits  by  establishing  relative 
weights  for  the  selection  criteria  and  relative 
ratings  for  the  alternatives.  This  paper 
briefly  reports  the  theory  behind  AHP  and 
its  potential  application  to  economic  analysis 
using  an  Air  Force  project  to  renovate  an 
aircraft  taxiway  as  an  example.  Despite 
several  potential  implementation  problems, 
this  decision  support  tool  should  foster  a 
more  complete,  systematic,  and  objective 
evaluation  of  acquisition  alternatives. 

INTRODUCTION 

When  selecting  a  project  or  set  of  projects 
to  acquire,  both  quantitative  and  qualitative 
factors  are  important.  In  "cost-benefit" 
analysis  the  benefits  as  well  as  the  costs  of 
alternative  investments  should  be 
considered.  Traditional  capital  investment 
models  that  employ  purely  economic 
selection  criteria,  such  as  net  present  value 
or  internal  rate  of  return,  have  been 
criticized  for  failure  to  adequately  include 
intangibles  [14,15]. 


This  problem  is  particularly  important  to  the 
Department  of  Defense,  where  qualitative 
noneconomic  factors  can  dominate.  Despite 
this,  there  is  little  guidance  regarding  how 
to  include  noneconomic  factors  in  what  the 
Department  of  Defense  narrowly  terms  an 
"economic  analysis."  Department  of 
Defense  Instruction  7041 .3  [7],  for  example, 
recognizes  the  need  to  assess  nonmonetary 
costs  and  benefits  across  alternatives,  but 
provides  little  procedural  guidance.  Air 
Force  Regulation  173-15  similarly  requires 
that  a  comparison  of  benefits  between 
alternatives  be  formally  documented  for  the 
decision  maker,  but  devotes  only  one 
paragraph  to  the  issue  [8:9]. 

A  multi-criteria  decision  model  is  needed 
that  can  combine  quantitative  and  qualitative 
factors  into  the  capital  investment  decision. 
The  model  should  not  replace  the  decision 
maker,  but  improve  the  capital  investment 
decision  by  a  systematic  consideration  of  all 
relevant  factors.  One  such  model  that  has 
received  significant  attention  and  broad 
application  is  Saaty’s  "Analytic  Hierarchy 
Process"  [24,28]. 

This  paper  describes  the  Analytic  Hierarchy 
Process  and  provides  an  example  using  data 
from  an  Air  Force  project  to  renovate  an 
aircraft  taxiway.  Limitations  and  potential 
implementation  problems  are  described. 
The  purpose  of  the  paper  is  to  inform  the 
reader  of  a  viable  tool  for  systematically 
including  nonquantifiable  factors  in  an 
economic  analysis. 
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THE  ANALYTIC  HIERARCHY 
PROCESS 

The  Analytic  Hierarchy  Process  (AHP)  is  a 
rational  and  systematic  approach  for  finding 
a  solution  to  a  problem.  The  method  allows 
decision  makers  to  partition  large 
unmanageable  problems  into  smaller  parts 
that  are  easier  to  handle.  It  provides 
decision  makers  with  the  ability  to  include 
qualitative  and  quantitative  criteria  to  form 
a  rating  for  each  of  the  alternatives.  These 
ratings  may  then  be  used  as  a  basis  for 
project  selection. 

Four  steps  are  used  to  solve  a  problem  with 
AHP:  (1)  build  a  decision  "hierarchy"  by 
breaking  the  general  problem  into  individual 
criteria;  (2)  gather  relational  data  for  the 
decision  criteria  and  alternatives;  (3) 
estimate  the  relative  weights  of  the  decision 
criteria  and  alternatives  using  the 
"proportional  method"  or  the  "eigenvalue 
method;"  and  (4)  aggregate  the  weights  of 
the  criteria  and  alternatives  into  a  vector  of 
ratings  for  the  alternatives,  which  can  be 
used  to  rank  the  alternatives. 

These  steps  are  perhaps  best  understood 
with  an  example.  Here,  data  from  an  Air 
Force  economic  analysis  are  used.  More 
elaborate  examples  of  using  AHP  in  capital 
acquisition  problems  are  available  elsewhere 
[10,22]. 

The  decision  problem  was  the  repair  or 
replacement  of  an  aircraft  taxiway  [2]. 
There  were  three  alternatives:  Alternative  A, 
repair  the  old  taxiway;  Alternative  B, 
construct  a  new  taxiway  with  material 
recycled  from  the  old  taxiway;  and 
Alternative  C,  construct  a  new  taxiway  and 
dispose  of  the  materials  from  the  old 
taxi  way. 

The  selection  criteria  were  (1)  the  cost  of 


each  alternative,  termed  "Cost";  (2)  the 
ability  of  each  alternative  to  sustain  aircraft 
weight,  termed  "Weight";  and  (3)  the 
minimization  of  taxiway  congestion,  termed 
"Congestion."  The  first  criterion  was 
quantitative,  and  based  on  estimates  of  a 
cost  analyst;  the  other  two  were  qualitative, 
and  based  on  the  judgment  of  the  decision 
maker. 

Step  1 .  As  illustrated  in  Figure  1,  the 
"hierarchy"  is  simply  a  tiered  framework  of 
decision  elements  related  to  the  problem, 
with  a  statement  of  the  overall  objective  at 
the  top  and  the  alternatives  at  the  bottom. 
Between  the  top  and  bottom  levels  are 
relevant  attributes  of  the  decision  problem, 
such  as  the  selection  criteria.  The  number 
of  tiers  depends  on  the  complexity  of  the 
problem  and  the  capacity  of  the  decision 
maker(s)  to  make  pairwise  comparisons 
between  the  elements  at  each  level. 


Step  2.  In  this  step,  relational  data  for 
comparing  the  alternatives  are  generated. 
Generating  quantitative  data,  such  as  the 
cost  of  each  alternative  is  straight  forward. 
In  an  economic  analysis,  costs  are  in 
constant  dollars  and  discounted  using  a  real 
10  percent  rate.  Generating  qualitative  data 
typically  requires  the  decision  maker  to 
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make  pairwise  comparisons  of  elements  at 
each  level  in  the  hierarchy  regarding  the 
contribution  made  to  the  objective  in  the 
next  higher  level.  In  the  taxiway  example, 
the  importance  of  each  criterion  to  the 
taxi  way,  and  the  relative  ratings  between 
alternatives  for  each  criterion  needed  to  be 
established. 

In  AHP  a  ratio  scale  of  real  numbers  from 
1  to  9  is  used  to  systematically  assign  the 
preferences.  In  the  taxiway  example,  the 
cost  criterion  was  considered  twice  as 
important  as  the  weight  criterion  and  five 
times  more  important  than  the  congestion 
criterion.  To  facilitate  the  pairwise 
comparisons,  verbal  descriptions  of  the 
numerical  ratings  may  be  used.  In  either 
case,  AHP  software  will  systematically 
prompt  the  decision  maker  to  make  all  the 
pairwise  comparisons  [9,16].  For  each 
pairwise  comparison,  the  decision  maker  is 
asked  which  element  is  most  important, 
preferred,  or  likely,  and  by  how  much  or 
how  many  times.  Tables  1  and  2  detail  the 
results  of  the  pairwise  comparisons. 

As  shown  in  Table  1 ,  the  cost  criterion  was 
judged  to  be  three  times  more  important  to 
selecting  the  taxiway  than  was  the  weight 
criterion.  Similarly,  the  cost  criterion  was 
judged  to  be  five  times  more  important  to 
the  taxiway  decision  than  was  the  congestion 
criterion.  The  weight  criterion  was  judged 
to  be  twice  as  important  as  the  congestion 
criterion. 

In  Table  2,  relational  data  derived  from 
comparing  the  alternatives  are  shown  for 
each  criterion.  For  the  cost  criterion,  the 
present  value  cost  of  each  alternative  is 
listed.  For  the  other  criteria,  pairwise 
comparisons  between  alternatives  were 
necessary.  The  relational  data  in  the  table 
reflect  the  decision  maker’s  preferences 
between  the  alternatives  with  respect  to  each 


criterion,  and  are  interpreted  the  same  way 
as  in  Table  1.  With  respect  to  the 
congestion  criterion,  for  example. 
Alternative  A  was  judged  to  be  the  most 
desirable. 

Tables  1  and  2  also  show  "inconsistency 
ratios.”  In  the  process  of  making  the 
pairwise  comparisons,  inconsistent 
judgments  are  possible.  In  the  taxiway 
example.  Cost  was  judged  to  be  more 
important  than  Weight,  and  Weight  was 
judged  to  be  more  important  than 
Congestion.  Accordingly,  Cost  should  be 
judged  more  important  than  Congestion. 
Otherwise,  the  comparison  would  be 
inconsistent. 

The  inconsistency  ratio  quantifies  an 
inconsistent  comparison.  An  inconsistency 
ratio  of  zero  reflects  perfect  consistency, 
which  is  difficult  to  achieve  when  there  are 
many  comparisons.  For  n  alternatives,  n(n- 
l)/2  comparisons  are  required  for  each 
criterion.  With  eight  alternatives,  for 
example,  28  pairwise  comparisons  would  be 
necessary  for  each  qualitative  criterion.  The 
inconsistency  ratio  is  thus  rather  useful 
feedback  to  improve  the  accuracy  of  the 
pairwise  comparisons. 

According  to  Saaty,  small  inconsistency 
ratios  do  not  drastically  affect  the  ratings  by 
AHP.  Based  on  experiments  reported  by 
Saaty  [20]  and  Vargas  [23],  a  "rule  of 
thumb"  is  recommended:  when  the 

inconsistency  ratio  is  less  than  .1,  the 
inconsistent  comparison  is  acceptable. 
When  the  ratio  is  larger  than  .1,  AHP 
software  will  prompt  the  decision  maker  to 
redo  the  comparisons.  Each  time  the 
pairwise  evaluations  are  re-accomplished,  a 
new  inconsistency  ratio  is  computed.  The 
process  of  revising  the  pairwise  judgments 
can  help  the  user  to  discover  illogical 
thinking  and  generate  insight  into  the 
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TABLE  1 

RELATIONAL  DATA  FROM  PAIRWISE  COMPARISONS  OF  TAXIWAY  CRITERIA 


Criteria 


Cost 


Weight 


Congestion 


Inconsistency  ratio  =  0.003 

TABLE  2 

RELATIONAL  DATA  FROM  PAIRWISE  COMPARISONS  OF  ALTERNATIVES 


Criterion 

Alternative  A 

Alternative  B 

Alternative  C 

Cost 

$5M 

$4M 

$3M 

Inconsistency  ratio  =  0.000 


Weight 


Alternative  A 


Alternative  B 


Alternative  C  5 


Inconsistency  ratio  =  0.003 


Alternative  A  Alternative  B  Alternative  C 


Congestion 


Alternative  A 


Alternative  B 


Alternative  C 


Inconsistency  ratio  =  0.000 


Alternative  A  Alternative  B  Alternative  C 


6 


TABLE  3 

NORMALIZED  WEIGHTS  AMD  RATINGS 


Alternative  Ratin 


Alternative  A  Alternative  B  Alternative  C 


.319 


.309 


.  118 


.291 


Criteria  (Weight) 


Cost 


6 


.230) 


Overall  rating  .276 


Inconsistency  ratio  =  0.000 


.426 


.582 


.  177 


.433 
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decision  problem. 

Step  3.  Based  on  the  relational  data 
developed  in  step  two,  two  methods  are  used 
to  assign  relative  weights  to  the  criteria  and 
relative  ratings  to  the  alternatives. 
According  to  Saaty,  a  "proportional  method" 
is  used  for  quantitative  data,  such  as  for  the 
cost  criterion,  and  an  "eigenvalue  method" 
is  appropriate  for  qualitative  data  generated 
by  the  pairwise  comparisons. 

In  the  proportional  method,  relative  weights 
are  assigned  according  to  the  amount  that 
each  alternative  contributes  to  the  sum  of  the 
values  of  all  of  the  alternatives  for  a 
particular  criterion.  Because  less  cost  is 
preferred  to  more  cost.  Alternative  C 
received  the  highest  rating  under  the  cost 
criterion.  In  the  eigenvalue  method,  an 
eigenvector  is  estimated  from  a  matrix  of 
the  pairwise  comparisons  to  calculate  the 
relative  ratings  for  each  alternative.  A 
detailed  description  of  this  method  is 
available  elsewhere  [17,18,19,20].  For 
either  method,  the  relative  weights  or  ratings 
are  normalized  to  add  to  unity. 

Table  3  shows  the  resulting  vector  of 
relative  weights  assigned  to  the  three  criteria 
and  the  matrix  of  relative  ratings  assigned  to 
the  alternatives  based  on  these  methods. 
The  cost  criterion  was  judged  to  be  the  most 
important  to  the  taxiway  decision,  with  a 
weighting  of  .648.  The  weight  and 

congestion  criteria  have  relative  weights  of 
.230  and  .122,  respectively.  For  the 
congestion  criterion.  Alternative  A  had  the 
highest  rating  of  .706;  for  the  weight  and 
cost  criteria,  Alternative  C  had  the  highest 
ratings. 

Step  4.  In  the  final  step,  the  relative 
weights  and  ratings  are  combined  into  a 
single  vector,  reflecting  the  overall  ratings 
of  the  alternatives.  The  vector  of  ratings  for 


the  alternatives  is  determined  by  a  series  of 
matrix  multiplications  beginning  at  the 
second  level  and  ending  at  the  lowest  level 
of  the  hierarchy.  In  the  taxi  way  example, 
the  ratings  of  the  alternatives  were 
calculated  by  multiplying  the  vector  of 
criteria  weights  against  the  matrix  of  relative 
ratings  to  yield  an  overall  rating  of  .276  for 
Alternative  A,  .291  for  Alternative  B,  and 
.433  for  Alternative  C.  Accordingly, 
Alternative  C  is  the  preferred  alternative. 

IMPLEMENTATION  ISSUES 

Having  reviewed  the  basics  of  AHP,  a  brief 
consideration  of  issues  involving  the  use  of 
AHP  in  an  economic  analysis  is  appropriate. 
The  following  listing  is  not  comprehensive, 
but  is  intended  to  highlight  issues  and  direct 
the  reader  to  the  relevant  literature. 

1.  The  eigenvalue  method  for  establishing 
relative  weights  is  computationally  difficult. 
"Without  a  familiarity  with  matrix  algebra, 
the  method  is  incomprehensible. 
Fortunately,  the  computations  are  made 
transparent  by  AHP  software  such  as  Expert 
Choice  [9],  or  Automan  [16].  Once  the 
hierarchy  is  established,  the  software  will 
systematically  lead  the  decision  maker 
through  the  necessary  pairwise  comparisons 
to  establish  the  weights  and  ratings.  After 
each  set  of  comparisons  is  completed,  the 
software  will  also  provide  information 
regarding  the  consistency  of  the  judgments 
by  reporting  the  inconsistency  ratio,  and 
prompt  the  user  to  redo  the  judgments  as 
appropriate.  Once  accomplished,  the 
software  will  compute  the  ratings. 

2.  The  ratio  scale  used  in  AHP  has  been 
criticized  as  inappropriate.  Saaty  [20] 
argues  that  the  intensity  of  preferences 
between  alternatives  can  be  expressed  using 
a  ratio  scale;  others  disagree,  claiming  that 
an  interval  scale  is  appropriate.  Harker  and 
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Vargas  [1 1]  review  this  issue  and  rebut  the 
criticism. 

3.  Saaty  [20]  has  warned  against  combining 
costs  with  benefits  in  one  hierarchy,  and 
recommended  separate  hierarchies  for  each. 
The  resulting  ratings  are  then  combined  as 
benefit-cost  ratios  to  establish  a  ranking 
suitable  for  selecting  projects.  Even  if  this 
is  done,  there  are  problems  with  the  use  of 
benefit-cost  ratios  derived  from  AHP  to 
select  capital  investment  projects  [5,6]. 
Generally,  the  problem  is  using  benefit-cost 
ratios  for  selecting  projects,  and  not  the 
method  of  establishing  the  ratios.  The 
capital  budgeting  literature  thoroughly 
addresses  this  more  general  problem  [4]. 

4.  A  problem  of  "rank  reversal"  has  been 
reported  [3,27].  Rank  reversal  occurs  when 
new  alternatives  are  added  to  the  hierarchy 
and  the  resulting  new  ranking  differs  from 
the  former  ranking.  If  the  new  alternatives 
provide  no  additional  information  on  the 
relative  rating  of  existing  alternatives,  then 
reversal  should  not  happen.  Marker  and 
Vargas  [11]  rebut  these  criticisms  as  an 
inappropriate  use  of  AHP,  and  suggest  a 
method  to  avoid  rank  reversals.  Schoner 
and  Wedley  also  explored  the  issue  and 
concluded  that  "Those  who  were  troubled  by 
rank  reversals  when  no  real  new  information 
is  added  should  be  reassured  that  AHP  can 
he  made  to  work  correctly"  [21:474]. 

5.  It’s  unclear  how  multiple  decision 
makers  or  evaluators  can  use  AHP.  Vargas 
and  Saaty  [25]  have  reported  a  few  cases. 
Graham  [10]  has  reported  the  use  of  AHP  in 
an  Air  Force  setting,  where  analysts  perform 
the  initial  analysis,  and  provide  it  to  a 
supervisor  for  consideration  and  possible 
revision. 

6.  In  a  typical  economic  analysis  conducted 
in  the  Department  of  Defen.se,  there  are 


often  significant  pressures  to  force  the 
analysis  to  favor  a  particular  alternative. 
Thus,  a  third-party  review  is  required  to 
identify  potential  bias.  It’s  clear  that  AHP 
can  also  be  manipulated  to  favor  a  project. 
AHP  software  can  facilitate  the  review  by 
thoroughly  documenting  the  decision 
problem  and  the  judgments  made  to  establish 
the  ratings.  In  addition,  AHP  software  can 
record  these  details  for  later  sensitivity 
analysis. 

CONCLUSION 

Despite  these  technical  issues,  AHP  has 
been  applied  to  capital  acquisition  problems 
[1,10,12,13,16,25,26],  and  is  consistent 
with  the  goals  of  economic  analysis. 

Economic  analysis  is  an  aid  to 
rational  choice  among  competing 
alternatives.  It  is  not  intended  to 
replace  the  judgment  of  the  decision 
maker  but  rather  to  aid  that 
Judgment.  ...  a  good  economic 
analysis  systematically  examines  and 
relates  cost,  benefits,  and  risks  of 
various  alternatives  [8:3]. 

AHP  facilitates  a  comprehensive  and 
rational  analysis  of  the  capital  acquisition 
problem.  Although  the  pairwise 
comparisons  required  in  AHP  may  seem 
tedious,  the  method  is  systematic  and 
comprehensive.  All  relevant  comparisons 
are  made.  When  the  comparisons  are 
illogical,  the  method  warns  the  decision 
maker  with  the  inconsistency  ratio. 

User  friendly  AHP  software  will  lead  the 
evaluator  through  all  the  pairwise 
comparisons,  and  compute  the  relative 
ratings  and  inconsistency  ratios.  The 
sofware  will  not,  however,  force  the  user 
into  revising  the  judgments.  AHP  is  a 
decision  support  tool;  it  does  not  replace  the 
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decision  maker. 


Other  important  requirements  of  economic 
analysis  are  sensitivity  analysis  and  thorough 
documentation  to  support  subsequent  third- 
party  review.  AHP  software  allows  the 
decision  maker  to  evaluate  the  sensitivity  of 
the  ranked  alternatives  to  the  weights 
assigned  to  the  criteria  and  alternatives.  Of 
course,  weights  can  be  easily  changed  and 
the  impact  of  the  change  on  the  ratings  is 
immediately  apparent.  In  addition,  the 
framework  of  the  hierarchy  and  the  weights 
are  recorded  for  subsequent  review.  It’s  a 
relatively  easy  matter  for  a  third  party  to  use 
AHP  software  to  evaluate  the  reasonableness 
another’s  judgments. 

In  short,  AHP  is  a  promising  tool  for 
facilitating  an  economic  anlysis.  Despite 
implementation  issues  and  occasional 
technical  objections,  AHP  has  received  wide 
application  in  multi-criteria  problems. 
Given  the  increasing  availability  of  personal 
computers  and  excellent  AHP  software,  such 
as  Expert  Choice  [9],  the  use  of  AHP  in  the 
capital  acquisition  problem  is  appropriate 
and  highly  recommended. 
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ABSTRACT 

The  National  Aeronautics  and  Space 
Administration  (NASA)  uses  the  Cost 
Plus  Award  Fee  (CPAF)  type  contract 
extensively  to  meet  program  goals 
associated  with  its  high  technology 
research  and  development  environ¬ 
ment.  Under  the  CPAF  type  contract, 
the  contractor  earns  fee  based  on  peri¬ 
odic  NASA  performance  evaluations. 
Recently  the  agency  has  sought  to 
improve  the  timeliness  and  reduce  the 
complexity  of  this  award  fee  perform¬ 
ance  evaluation  process. 

On  a  major  support  services  contract  at 
the  Lyndon  B.  Johnson  Space  Center 
(JSC),  substantial  improvements  have 
been  implemented  to  reduce  the  effort 
and  paperwork  involved  in  the  process 
and  also  decrease  the  time  it  takes  to 
inform  the  contractor  of  the  fee  deter¬ 
mination  award. 

The  improvements  discussed  in  this 
paper  involve:  (1)  consolidating  many 
similar  small  job  orders  into  larger 
ones  for  more  effective  control  and  cost 
avoidance;  (2)  elevating  the  lowest 
level  of  evaluation  to  the  division  level 
while  retaining  job  order  evaluation  as 
a  local  option;  (3)  training  NASA 
personnel  in  performance  evaluation  - 
this  training  included  both  the  publi¬ 
cation  of  the  Engineering  Support 
Contract  (ESC)  Management  Guide, 
which  documents  this  process,  and  the 
addition  of  an  award  fee  module  in  an 
existing  training  course;  and  (4)  invit¬ 
ing  the  contractor  to  make  an  oral 
presentation  of  its  self-evaluation. 


These  innovations,  which  strongly 
support  NASA’s  commitment  to  the 
continuous  improvement  aspect  of  total 
quality  management  (TQM),  are 
presently  under  consideration  as  a 
candidate  for  best  practices  in  tech¬ 
nical  contract  management  to  be 
shared  with  other  NASA  centers. 


INTRODUCTION 

NASA  is  heavily  involved  in  the 
planning,  development,  and  operations 
of  several  programs  supporting  our 
National  space  effort.  These  include 
the  Space  Shuttle,  Space  Station,  and 
Advanced  Programs.  The  resources 
necessary  to  support  these  programs 
come  from  diverse  combinations  of  civil 
service  and  contractor  organizations. 
Primary  contractor  support  for  these 
programs  comes  from  major  prime 
contractors  who  are  involved  in  hard¬ 
ware  development.  However,  NASA 
augments  its  technical  skills  through 
service  type  contracts  providing  sup¬ 
port  in  the  areas  of  engineering  and 
science  analysis,  testing,  simulation, 
and  integration,  as  well  as  facility 
operation  and  maintenance. 

JSC,  in  particular,  has  found  that  these 
augmented  technical  skills  have  been 
provided  effectively  under  a  CPAF  type 
contract.  One  such  CPAF  type  contract 
at  JSC  is  the  Engineering  Support 
Contract  (ESC).  The  ESC  is  a  high- 
technology,  level-of-effort  contract 
involving  some  2,200  Lockheed  Engi¬ 
neering  &  Sciences  Company  (LESC) 
personnel  who  provide  services  to 
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twenty  different  JSC  Divisions  and 
Offices.  These  JSC  organizations  have 
major  responsibilities  in  ongoing 
efforts  for  the  Space  Shuttle,  Space 
Station,  and  Advanced  Programs. 

Under  CPAF  provisions,  the  contractor 
is  awarded  a  fee  based  on  technical, 
management,  and  cost  performance. 
Recently  this  process  has  come  under 
the  scrutiny  of  senior  NASA  manage¬ 
ment  for  its  complexity  and  its  failure 
to  be  conducted  in  a  timely  manner. 

Darleen  A.  Druyun,  former  NASA 
Associate  Administrator  for  Procure¬ 
ment,  commented  on  this  problem 
in  an  October  1992  "CM  Interview” 
article  in  Contract  Management,  the 
National  Contract  Management 
Association  (NCMA)  periodical. 
Ms.  Druyun  was  responsible  for  lead¬ 
ing  a  number  of  initiatives  to  stream¬ 
line  the  NASA  procurement  process. 
One  of  her  first  initiatives  was  to 
improve  relations  between  the  con¬ 
tractors  and  NASA.  Contractors  had 
expressed  frustration  in  dealing  with 
NASA  on  contractual  matters.  "About 
76  percent  of  NASA’s  contracts  are 
CPAF,”  stated  Ms.  Druyun,  "so  it  was 
almost  a  safe  bet  that  one  of  industry’s 
chief  complaints  would  come  in  the 
area  of  CPAF  contracting.”  She  also 
mentioned  that  many  contractors  had 
told  her  that  "some  of  the  agency 
centers”  had  been  taking  "up  to 
4  months  to  process  payment  after  the 
board  met.” 

While  NASA  management  was  review¬ 
ing  its  award  fee  contracting  process, 
parallel  efforts  to  improve  this  process 
were  being  implemented  at  JSC  on  the 
ESC. 


THE  ENGINEERING  SUPPORT 
CONTRACT 

The  ESC  requires  periodic  evaluation 
of  contractor  performance.  Fee  dollars 
are  awarded  to  the  existing  contractor, 
LESC,  based  on  its  performance.  The 


ESC  Award  Fee  Plan  provides  a 
structure  whereby  LESC  is  evaluated 
on  a  quarterly  basis  and  fee  is  awarded 
on  a  semiannual  basis. 

Work  is  authorized  on  the  ESC  through 
the  use  of  fiscal  year  job  orders  issued 
by  JSC  Divisions  or  Offices.  This  level 
of  work  is  the  basis  of  cost  and  tech¬ 
nical  progress  reporting.  Summary 
level  report  rollups  follow  an  organiza¬ 
tional  breakdown  structure  (OBS). 
Figure  1  depicts  the  relationship 
between  the  functional  work  unit 
structure  and  the  OBS. 

Previous  performance  evaluations 
were  made  at  the  job  order  level  and 
then  summarized  to  the  Division  level; 
thence  to  the  Directorate  level;  and 
thence  to  an  ESC  level,  where  the  fee 
determination  was  made.  The  large 
number  of  small  job  orders  made  this 
process  very  time  consuming  and 
inefficient.  As  a  direct  result  of  the 
vast  amount  of  time  and  paperv/ork 
inherent  in  the  process,  the  fee 
determination  to  LESC  was  taking 
over  3  months. 

A  research  effort  was  initiated  to 
determine  methods  to  improve  this 
process.  Care  was  taken  not  to  degrade 
the  Government’s  need  for  a  perform¬ 
ance  evaluation  tool.  Likewise,  a  time¬ 
ly  contractor  performance  feedback 
mechanism  needed  to  be  preserved  that 
facilitated  corrective  action  prior  to  the 
next  evaluation  period. 


FACT  FINDING 

A  review  of  the  ESC  Award  Fee  Plan 
and  the  applicable  JSC  Management 
Directive  was  conducted  to  determine 
the  real  requirements.  The  require¬ 
ments  in  the  contract  were  ( lithe 
contractor  had  to  be  evaluated  quar¬ 
terly  by  a  Performance  Evaluation 
Committee  (PEC);  (2)  a  Performance 
Evaluation  Board  (PEB)  had  to  review 
a  6-month  evaluation  report  submitted 
by  the  PEC;  (3)  the  PEB  submitted  its 
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evaluation  to  a  Performance  Determi¬ 
nation  Official  (PDO),  a  senior  NASA 
official;  and  (4)  the  PDO  made  the 
award  fee  determination.  The  PDO 
also  had  to  give  consideration  to  infor¬ 
mation  submitted  by  the  contractor 
within  15  days  after  the  award  fee 
period  ended. 

In  addition  to  what  was  in  the  contract, 
further  requirements  were  contained 
in  a  JSC  Management  Directive,  "Per¬ 
formance  Evaluation  and  Notification 
Procedures  for  Award  Fee  Contracts,” 
issued  November  23,  1988,  by  JSC 
Director  Aaron  Cohen.  The  Directive 
states  that  the  PEC  is  required  to  have 
its  evaluation  information  to  the  PEB 
secretary  no  later  than  30  days  after 
the  evaluation  period  ends.  The  PEC 
secretary  has  15  days  following  the  30 
to  resolve  any  corrections  to  the  evalu¬ 
ation  information  with  the  PEC  chair¬ 
person.  And  finally,  the  PEB  is 
required  to  have  scheduled  and  re¬ 
viewed  the  evaluation  information 
submitted  by  the  PEC  within  15  days 


after  the  PEB  secretary  has  reviewed 
the  information. 

During  the  research,  a  process  flow  of 
the  performance  evaluation  was  pre¬ 
pared.  The  flow  revealed  a  substantial 
amount  of  lower  level  backup  (paper) 
was  being  generated  to  support  the 
PEC’s  contract  level  recommendation 
to  the  PEB.  Figure  2  depicts  the  per¬ 
formance  evaluation  process,  identify¬ 
ing  areas  that  were  eliminated. 

A  time  line  was  also  constructed  that 
depicted  the  actual  time  from  the  end 
of  the  performance  evaluation  period  to 
actual  fee  determination.  The  six 
periods  measured  averaged  70  days, 
with  the  highest  being  99  days. 
Figure  3  depicts  historical  processing 
for  those  evaluations. 


JOB  ORDER  CONSOLIDATION 

While  the  performance  evaluation  pro¬ 
cess  was  being  reviewed,  a  TQM  effort 
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was  also  in  process  to  reduce  the 
excessive  number  of  job  orders  -  in 
particular,  the  small  duplicate  ones. 

Over  the  past  several  years  as  funds 
became  more  limited,  the  need  arose  to 
plan  and  track  line  items  at  smaller 
values.  Job  orders  were  written  to 
match  these  smaller  funding  amounts. 
Quite  often  these  small  job  orders  were 
functionally  redundant.  As  many  as 
813  job  orders  were  in  effect  for  fiscal 
year  1990  on  the  ESC.  This  was  2.5 
equivalent  persons  per  job  order  that 
year  alone. 

Discussions  were  held  with  the  entire 
ESC  management  team.  Technical, 


business,  and  contract  managers  from 
NASA  were  involved,  as  well  as 
contractor  management  representa¬ 
tives.  The  result  of  this  team  effort, 
achieved  for  fiscal  year  1991,  was  a 
reduction  from  813  to  500  job  orders. 
The  savings  due  to  the  reduction  in 
paperwork,  data  processing,  reporting, 
and  performance  evaluation  effort 
were  obvious.  An  even  better  improve¬ 
ment  was  realized  in  fiscal  year  1992, 
when  the  number  of  job  orders  was 
reduced  to  165  with  the  implemen¬ 
tation  of  another  TQM  initiative  -  the 
Simplified  Project  Control  System. 
The  number  of  equivalent  persons  per 
job  order  increased  to  12.3  in  this  year. 
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Figure  3.- ESC  performance  evaluation  process  cycle  time  history. 


PROCESS  IMPROVEMENT 

An  analysis  of  the  ESC  Performance 
Evaluation  Process  Flow  revealed  that 
any  significant  reduction  to  the  process 
would  have  to  be  made  at  the  lowest 
level  where  some  250  NASA  job  order 
monitors  evaluated  some  500  job 
orders.  No  changes  could  be  made  to 
the  organization  structure  where  the 
summary  rollups  occurred. 

The  NASA  Technical  Manager 
proposed  raising  the  lowest  level  of  the 
evaluation  that  would  be  delivered  to 
the  PEC  to  an  overall  evaluation  at  the 
Division  level,  thus  removing  the  for¬ 
mal  requirement  to  deliver  an  evalua¬ 
tion  by  job  order  to  the  PEC.  However, 
each  Division/Office  would  retain  dis¬ 
cretion  regarding  the  internal  proce¬ 
dures  used  to  arrive  at  the  Division 
level  evaluation.  This  suggestion  was 
briefed  to  the  Division/Office  level 
managers.  The  change  was  overwhel¬ 
mingly  accepted  and  implemented  for 


the  evaluation  period  in  progress  at  the 
time. 

The  savings  at  the  Division  level,  with 
no  degradation  to  the  process,  were 
immediate.  Two  Division  Chiefs  wrote 
letters  of  commendation  to  the  NASA 
Technical  Manager  for  taking  the 
initiative  to  delete  the  need  for 
unnecessary  requirements. 

In  concert  with  raising  the  lowest  level 
of  evaluation,  all  of  the  steps  in  the  pro¬ 
cess  were  put  to  the  acid  test  and  exam¬ 
ined  for  "value  added”  contributions. 
This  resulted  in  the  elimination  of 
adjective  rating  sheets  and  the  re¬ 
duction  in  the  size  of  the  presentation 
package  used  to  brief  both  the  PEC  and 
the  PEB. 

Other  process  improvements  involved 
realignment  of  certain  activities  to 
enhance  the  effectiveness  and  effi¬ 
ciency  of  the  performance  evaluation 
process.  For  instance,  the  original 
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contract  specified  performance  evalua¬ 
tion  periods  in  6-month  intervals  from 
the  start  date  of  the  contract.  This 
resulted  in  evaluation  periods  that  did 
not  coincide  with  the  normal  Govern¬ 
ment  fiscal  year  operating  cycle  on 
which  the  contract  is  conducted. 
Realignment  of  the  evaluation  periods 
to  coincide  with  the  Government  fiscal 
year  allowed  the  utilization  of  informa¬ 
tion  routinely  reported  instead  of 
information  which  had  to  be  reformat¬ 
ted  to  fit  the  unique  evaluation  periods. 
Additionally,  original  contract  provi¬ 
sions  were  modified  to  revise  the 
requirements  for  the  contractor’s 
Quarterly  Progress  Report.  The 
reporting  level,  which  was  originally  at 
the  job  order  level,  was  elevated  to  the 
Division  level  to  coincide  with  the  level 
at  which  the  evaluation  was  now 
performed.  The  size  of  the  report  and 
the  management  preparation  time 
both  decreased  significantly,  and  the 
accelerated  delivery  schedule  allowed 
the  information  to  get  to  the  NASA 
Divisions  in  time  to  be  utilized  during 
the  contractor  performance  evaluation. 

The  new  process  has  been  used  three 
times  with  no  complaints  from  the 
contractor  or  NASA  evaluators.  The 
new  process  has  also  reduced  the  time 
usually  required  to  debrief  the  contrac¬ 
tor.  The  technical  level  of  detail  to 
require  corrective  action  is  still 
retained. 


NASA  TRAINING 

In  addition  to  the  large  volume  of 
NASA  technical  management  person¬ 
nel  involved  in  the  evaluation  process, 
a  large  turnover  of  these  personnel  was 
being  experienced  that  added  to  the 
training  challenge. 

A  course  had  been  developed  by  the 
contractor,  and  paid  for  by  NASA, 
entitled  The  Job  Order/Project  Control 
System.  Initially  geared  for  contractor 
personnel,  the  course  was  modified  for 


NASA  attendance,  and  an  Award  Fee 
Process  module  was  added  to  the 
curriculum.  This  new  module  was 
instructed  by  the  NASA  Technical 
Manager.  However,  because  this  train¬ 
ing  course  was  considered  familiari¬ 
zation  training  on  a  volunteer  basis, 
not  everyone  attended.  Based  on  low 
attendance  by  NASA  personnel, 
another  method  was  needed  to  reach 
all  the  players. 

The  NASA  Technical  Manager  created 
the  idea  for  the  ESC  Management 
Guide,  which,  among  other  things, 
documents  the  evaluation  process. 
Evaluation  forms  are  included,  as  well 
as  the  purp.se  and  composition  of  the 
committee  and  the  board.  The  ESC 
Management  Guide  is  w  dely  distrib¬ 
uted  and  is  revised  and  distributed 
usually  at  the  beginning  of  each  fiscal 
year  or  as  needed. 


CONTRACTOR  PRESENTATION 
OF  SELF-EVALUATION 

The  ESC  Award  Fee  Plan  provides  the 
contractor  with  the  opportunity  to  sub¬ 
mit  a  self-evaluation  at  the  conclusion 
of  an  evaluation  period.  Traditionally, 
this  self-evaluation  was  submitted  in 
written  format,  addressing  cost  per¬ 
formance  and,  more  recently,  man¬ 
agement  and  productivity.  Since  the 
self-evaluation  was  in  written  format 
only,  it  provided  no  formal  mechanism 
for  NASA-contractor  interaction  in 
regard  to  what  was  submitted.  An 
occasional  telephone  call  about  a  line 
item  was  not  an  adequate  interchange. 
The  NASA  PEC  chairperson  offered 
LESC’s  program  manager  the  oppor¬ 
tunity  to  present  the  contractor  self- 
evaluation  in  the  form  of  a  briefing. 
This  briefing  has  been  in  existence  for 
the  last  two  6-month  evaluation 
periods,  with  PLC  members  attending 
on  a  voluntary  basis.  Because  it  has 
effectively  fostered  verbal  interchange 
on  major  issues,  it  has  proven  to  be 
quite  beneficial,  in  achieving  a  better 


138 


understanding  of  both  the  performance 
and  the  operational  drivers. 


WHAT  NEXT? 

Although  significant  progress  in 
improving  the  award  fee  evaluation 
process  has  been  achieved  on  the  ESC, 
NASA  and  JSC  have  been  slower,  on 
the  average,  in  providing  feedback  and 
paying  fee  on  CPAF  type  contracts. 
Data  submitted  to  NASA  Head¬ 
quarters  in  October  1992  indicated 
that  JSC  averaged  118  calendar  days 
from  the  end  of  the  evaluation  period 
until  the  payment  of  the  award  fee  to 
the  contractor.  The  118  days  consisted 
of  92  days  for  fee  determination  and  an 
additional  26  days  for  payment  of  the 
award  fee.  In  a  letter  dated  December 
30,  1992,  from  the  NASA  Administra¬ 
tor  to  the  Directors  of  the  field  installa¬ 
tions,  Daniel  S.  Goldin  indicated  he 
was  "disappointed  to  see  the  length  of 
time  it  takes  NASA  to  provide  a  final 
evaluation  of  performance  to  contrac¬ 
tors  after  the  end  of  the  period,  and 
then  to  pay  them  the  actual  fee 
awarded.”  In  the  same  letter,  Goldin 
established  an  objective  of  45  calendar 
days  from  the  end  of  the  evaluation 
period  to  the  issuance  of  the  final  fee 
determination  and  15  calendar  days 
from  the  fee  determination  to  the 
payment  of  fee,  for  a  total  of  60  days  to 
complete  the  entire  process. 

In  response  to  Goldin’s  objective,  JSC 
Acting  Director  Paul  J.  Weitz  has 
recently  set  more  specific  objectives  to 
help  achieve  the  60-day  goal  by 
March  31,  1993.  One  of  the  more 
critical  objectives  involves  shortening 
the  process  times  of  PEC  and  PEB 
activities.  The  PEC  activities  will 
have  to  be  completed  within  20  calen¬ 
dar  days  from  the  end  of  the  evaluation 
period  and  the  PEB  activities  within 
an  add'  Jonal  10  days.  The  fee  determi¬ 
nation  will  then  have  to  be  made 
within  15  days  of  the  PEB  recommen¬ 
dation  and  the  payment  of  award  fee 


not  later  than  15  days  thereafter. 
Under  these  objectives,  the  entire 
process  would  be  completed  within 
60  calendar  days  after  the  end  of  the 
evaluation  period. 

Although  the  average  time  for  per¬ 
formance  determination  and  award  fee 
payment  on  the  ESC  has  been  reduced 
to  70  days  (versus  the  JSC  average  of 
118  days  cited  earlier),  further 
improvements  will  be  necessary  to 
achieve  the  60-day  objective.  Empha¬ 
sis  will  be  placed  on  documenting 
performance  drivers  throughout  the 
evaluation  period  so  that  formalization 
and  submittal  of  the  documentation  at 
the  end  of  the  period  will  be  a  rela¬ 
tively  short,  simple  task  and  will  allow 
the  PEC  activities  to  be  completed 
earlier.  Additionally,  PEB  activities 
could  be  shortened  by  prescheduling 
the  PEB  meeting  and  eliminating  or 
rephasing  the  PEB  Secretary’s  review 
of  the  PEC’s  report  to  the  PEB,  which 
currently  occurs  prior  to  the  PEB 
meeting.  Finally,  successfully  imple¬ 
menting  the  new  15-day  objective  for 
PDO  activities  should  ensure  that  the 
entire  process  is  completed  on  a  more 
timely  basis. 


SUMMAKY 

The  techniques  which  have  been  and 
will  be  used  to  improve  the  ESC  Award 
Fee  Performance  Evaluation  Process 
are  fairly  straightforward  and  prob¬ 
ably  apply  to  other  managerial 
practices.  The  following  steps  are 
presented  for  reader  consideration: 

1.  Determine  what  the  "real”  require¬ 
ments  are,  and  create  a  work  flow 
for  that  process.  It’s  difficult  to 
improve  a  process  before  it’s  docu¬ 
mented. 

2.  Perform  a  "value  added”  test 
against  each  activity.  Eliminate 
rubber  stamp  and  unnecessary 
activities. 
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3.  Don’t  be  afraid  to  take  risks  to 
eliminate  paper,  raise  approval 
levels,  and  get  to  the  heart  of 
absolute  requirements. 

4.  Empower  people  to  make  and 
implement  change. 
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MODELING  AND  SIMULATION  FOR  SOURCE  SELECTION  BOARDS 


Mr.  Dennis  M.  Clark  11,  SPA  WAR  42 


Abstract 

This  research  effort  details  a 
modeling  and  simulation  effort  which 
supported  the  evaluation  of  proposals  for  a 
Fiber  Optic  Data  Bus.  The  intent  was  to 
model  each  contractor’s  proposal  and 
compare  the  various  proposals’  technical 
performance.  The  effort  would  allow  an 
objective  comparison  -  based  on 
quantifiable  data  -  of  the  proposals  to 
augment  the  Judgment  of  the  Technical 
Evaluation  Team.  This  research  describes 
the  results  of  these  efforts  and  lessons 
learned  in  relation  to  utilizing  these 
modeling  and  simulation  techniques  for 
source  selection  of  this  nature. 

Task  Description 

The  Center  for  Advanced  Concepts 
Development  (CACD)  was  asked  to 
participate  in  the  Fiber  Optic  Data  Bus 
(FODB)  proposal  evaluation  by  providing 
modeling  support  to  the  Technical 
Evaluation  Team  (TET).  The  intent  was  to 
model  each  contractor’s  proposal  and 
compare  the  various  proposals’  technical 
performance.  It  was  hoped  that  this  effort 
would  allow  an  objective  comparison  - 
based  on  quantifiable  data  -  of  the  proposals 
to  augment  the  expert  judgment  of  the  TET. 

The  Request  For  Proposal  (RFP) 
announced  that  each  proposal  would  be 
modeled  by  the  government  in  the  OPNET“ 
simulation  environment.  Furthermore,  each 
contractor  was  required  to  complete  a 


detailed  Data  Information  Form  in  the  RFP 
and  include  it  with  their  proposals. 

Proposal  offerors  were  required  to 
submit  bus  architectures  that  operated  in 
two  distinct  throughput  ranges.  A  Low- 
End  architecture  was  to  address  throughput 
requirements  in  the  lOOMb/sec  -  800Mb/sec 
range.  The  High-End  architecture  was  to 
address  throughput  requirements  in  the 
800Mb/sec  - 1 .2Gb/sec  range 

Figure  1  illustrates  the  procedure 
developed  for  the  modeling  effort.  CACD 
involvement  began  several  months  before 
the  RFP  was  issued.  CACD  personnel 
reviewed  the  RFP,  recommended  many 
changes,  developed  the  Data  Information 
Form,  and  assisted  with  the  development  of 
the  Measures  of  Effectiveness  (MOEs)  for 
the  modeling  effort.  To  expedite  model 
development,  in  what  was  anticipated  to  be 
a  limited  amount  of  time,  the  most  likely 
architectures  (star,  FDDI,  and  Token  Ring) 
were  identified  and  “skeleton”  models  were 
developed  prior  to  the  delivery  of  the  RFP 
responses.  The  intent  was  to  flesh  out  the 
skeletons  with  details  provided  by 
contractor  Data  Information  Forms  and 
complete  the  modeling  effort  rapidly  during 
the  evaluation  phase. 

The  strategy  then,  was  to  complete 
skeleton  models  with  details  provided  by  the 
offerors,  run  the  models  with  varying  data 
rates  and  packet  sizes,  and  compare  the  data 
throughputs.  Data  throughput  measured 
against  varying  data  rates  and  packet  sizes 
was  the  adopted  MOE. 
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Procedure 


Figure  I 


ProDosal 

Architecture 

Modelinq  Effort 

#1  LOW-END 

Star 

Model  Developed 

#1  HIGH-END 

MUX 

None  -  Protocol  Independent 

Ml  LOW-END 

Ring 

None  -  Insuificient  Data 

Ml  HIGH-END 

Ring 

None  -  Insufficient  Data 

M3  LOW-END 

Star 

Model  Developed 

M3  HIGH-END  1 

Star 

Model  Developed 

#3  HIGH-END2 

MUX 

None  -  Protocol  Independent 

#4  HIGH-END 

Star 

None  -  Insufficient  Data 

M4  LOW-END 

Star 

None  -  Insufficient  Data 

M5  LOW-END 

Ring 

Model  Provided 

#5  HIGH-END 

_ Eins _ 

Model  Provided 

Table  I 
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A  month  before  the  actual 
evaluation,  a  “mock”  evaluation  was 
conducted  to  identify  problems,  correct 
them,  and  refine  the  tools  and  procedures 
necessary  to  complete  the  modeling  effort. 
The  mock  evaluation  was  successful; 
however,  members  of  the  TET  announced 
that  only  two  weeks  could  be  allowed  for 
the  modeling  effort  vice  the  four  to  six 
weeks  shown  in  figure  1.  At  the  time,  this 
reduction  in  time  was  not  of  concern  since 
only  a  limited  number  of  proposals  was 
anticipated. 

Modeling:  Summary  and  Discussion 
Summary 

Table  1  summarizes  both  the 
proposals  and  modeling  effort.  Figures  2-4, 
which  follow,  show  the  technical 
performance  of  the  proposals  which  were 
modeled. 

Discussion 

There  were  a  total  of  five  responses 
to  the  RFP.  Included  in  these  five  responses 
were  eleven  proposals.  In  the  interest  of 
anonymity,  the  proposals  are  hereafter 
referred  to  by  number. 

Proposal  #1  Low-End 

The  proposed  architecture  for  this 
design  was  a  star  configuration  with  a 
maximum  of  32  nodes.  The  protocol,  also 
known  as  the  High  Speed  Data  Bus 
(HSDB),  is  based  on  the  linear  token 
passing  bus.  This  variant  employs  a  central 
controller.  The  controller  uniformly 
distributes  tokens  to  the  Bus  Interface  Units 
(BIUs). 


The  proposal  claimed  a  bandwidth 
of  200  MB/sec  which  the  model  verified. 
This  bandwidth  was  achieved  with  a 
saturated  bus  whose  saturation  rate  was 
calculated  to  be  760  packets/sec/node  for  32 
nodes.  The  throughput  for  the  remaining 
packet  sizes  was  consistent  with  the  4b/5b 
encoding  scheme. 

Further  analysis  with  varying  packet 
sizes  and  data  rates  revealed  that  this  design 
was  technically  capable  of  supporting  the 
low  end  data  rates. 

Proposal  #1  High-End 

The  proposed  architecture  for  this 
design  was  a  high-speed  multiplexer.  Since 
the  approach’s  protocol  demonstrated  no 
perturbations  as  data  rates  varied  and  the 
thrust  of  the  modeling  strategy  was  to 
measure  these  perturbations,  this  proposal 
and  others  with  similar  architectures  was 
not  modeled. 

Proposal  #2  Low-End 

There  were  several  difficulties 
involved  with  attempting  to  construct  a 
model  with  this  proposal.  It  combined  a 
Synchronous  Optical  NETwork  (SONET) 
protocol  with  an  FDDI-1  overlay.  In  this 
scheme,  the  FDDI  overlay  functioned  as  the 
Asynchronous  Transfer  Mode  (ATM)  layer. 
While  the  CACD  modeling  library  contains 
an  FDDI  model,  it  does  not  contain  a 
SONET  model.  The  addition  of  a  SONET 
model  of  sufficient  fidelity  proved  to  be 
outside  the  time  constraints  of  this  project. 
This  proposal  was  one  of  several  proposals 
that  made  use  of  the  SONET  protocol.  The 
appearance  of  this  protocol  was  surprising 
to  the  CACD  and  the  TET.  SONET  had 
not  been  among  those  architectures  and 
protocols  that  were  identified  and  modeled 
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during  the  preparatory  phase.  To  add  to  the 
difficulty,  the  proposal  failed  to  adequately 
explain  the  integration  of  the  SONET  and 
FDDI  protocols.  Combined,  these 
difficulties  prevented  modeling  this 
proposal. 

Proposal  High-End 

This  proposal  combined  a  SONET 
protocol  with  a  ring  topology.  The 
proposal,  like  its  low-end  partner,  failed  to 
adequately  describe  its  implementation  of 
SONET.  That  failure,  in  addition  to  the 
CACD’s  lack  of  experience  with  the 
SONET  protocol,  also  removed  this 
proposal  from  modeling  consideration. 

Proposal  #3  Low-End 

The  proposed  architecture  for  this 
design  was  a  star  configuration  with  a 
maximum  of  32  nodes.  Its  protocol 
consisted  of  a  proprietary  token-passing 
scheme  with  a  central  controller  which 
uniformly  distributed  tokens. 

The  proposal  claimed  a  bandwidth 
of  300  Mb/sec;  however,  this  claim  could 
not  be  substantiated  with  the  CACD’s 
model.  The  model  demonstrated  a 
maximum  throughput  of  190  Mb/sec  using  a 
saturation  rate  of  1 144  packets/sec/node  for 
32  nodes.  Throughput  for  the  remaining 
packet  sizes  was  consistent  with  the 
demonstrated  maximum  rate  and  the  4b/5b 
encoding  scheme. 

This  design  consistently  fell  short  of 
its  bandwidth  claims  during  testing.  The 
model  was  run  using  a  range  of  packet  sizes 
and  data  rates.  Its  bandwidth  was 
consistent  with  a  single-stream  transmit  rate 
with  encoding. 


Proposal  Ui  High-End  j  (Two  Designs) 

The  proposed  architecture  for  this 
design  was  a  star  architecture  with  a 
maximum  of  32  nodes.  Its  protocol  was  a 
polling  scheme  which  distributed  SONET 
frames.  Since  the  proposal  fully  described 
its  implementation  of  SONET  -  particularly 
its  use  of  SONET  frames  -  the  proposal 
was  easily  modeled. 

The  proposal  claimed  a  bandwidth 
of  2.488  GB/sec,  the  maximum  SONET 
bandwidth.  CACD’s  model  verified  this 
claim  over  a  range  of  packet  sizes. 
Bandwidth  was  calculated  using  a  saturation 
rate  of  250  packets/sec/node  for  32  nodes. 
Since  there  was  no  encoding  scheme  used, 
the  throughput  for  the  various  packet  sizes 
was  consistent  with  the  maximum  rate. 

Additional  model  runs  using  a  range 
of  packet  sizes  and  data  rates  demonstrated 
that  the  design  can  support  all  high-end 
throughput  requirements. 

Proposal  #3  High-End2 

Another  multiplexing  scheme.  As 
stated  earlier  for  Proposal  #1  High-End,  a 
multiplexing  architecture  shows  no 
perturbations  as  data  rates  are  varied; 
therefore,  this  proposal  was  not  modeled. 

Proposal  #4  Low-End 

This  proposal  used  a  ring 
architecture  with  a  Time-Division  Demand 
Assignment  protocol.  Unfortunately,  the 
Data  Information  Form  and  the  text  of  the 
proposal  were  extremely  confusing.  We 
encountered  several  problems  because  of 
inconsistencies  between  the  text  and 
illustrations  and  with  inadequate 
descriptions  of  the  frame  formats.  After 
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stniggling  to  understand  the  proposal,  we 
enlisted  the  help  of  the  TET’s 
communication  engineering  staff.  They 
were  equally  confused  by  the  proposal’s 
descriptions.  The  inability  of  this  proposal 
to  clearly  define  its  protocol  removed  it 
from  modeling  consideration. 

Proposal  #4  High-End 

This  proposal  closely  paralleled  its 
Low-End  partner.  It  was  equally  confusing 
and  was  likewise  removed  from  modeling 
consideration. 

Proposal  #5  Low-End/High-End 

This  proposal  used  a  unique  design 
which  was  scalable  across  both  the  low-  and 
high-end  throughput  rates.  The  network 
topology  consisted  of  a  ring  with  a  rotating 
data  frame.  The  protocol  incorporated  a 
large  frame  (1800  bits)  with  assigned  slots 
for  each  BIU.  The  frame  rotated  around  the 
ring  from  station  to  station.  As  the  frame 
arrived  at  each  station  the  local  BIU  wrote 
information  in  its  assigned  slot.  Thirty-two 
ring  stations  were  used. 

The  throughput  rates  for  this 
architecture  were  dependent  on  the  rate  at 
which  the  frame  rotated  around  the  ring  and 
not  on  the  rate  at  which  the  BIUs  generated 
packets.  In  this  respect  it  was  similar  to  the 
high-end  MUX  submissions  in  proposals  #1 
and  #3 . 

A  complete  OPNET™  model  was 
included  as  part  of  the  required  Data  Sheet. 
Also  included  were  examples  of  expected 
output  from  the  model  runs.  Unfortunately, 
after  copying  the  included  model  and 
regenerating  the  output,  the  provided 
examples  could  not  be  duplicated.  In  those 
examples,  the  queues  were  filling  up  and 


emptying  at  a  steady  rate.  The  graphs 
generated  by  the  CACD’s  model  show  that 
the  queues  were  filling  up  at  a  rate  greater 
than  the  rate  at  which  they  were  being 
emptied.  Although  the  modeled  throughput 
was  shown  to  be  the  same  as  that  claimed, 
the  behavior  of  the  queues  was  of  concern. 


Benefits 

The  benefits  of  this  unique  approach 
to  source  selection  were  as  follows: 

(1)  Verification  of  proposal 
performance  claims. 

(2)  Quick  identification  of 
incomplete  or  immature 
designs. 

(3)  Lay  the  groundwork  for 
future  efforts. 

Verification 

Although  the  number  of  proposals 
that  were  completely  modeled  proved  to  be 
less  than  anticipated,  those  that  were 
modeled  provided  new  insight  to  verifying 
the  claims  made  in  the  proposals.  This 
project  was  sensitive  to  the  performance  of 
the  seiectvd  design  as  indicated  by  the 

primary  MOE  (throughput).  Therefore,  any 
quantifiable  comparison  of  designs  was 
welcome.  The  intent  was  to  provide  an 
objective  measurement  of  the  performance 
of  the  designs.  In  this  way  not  only  could 
designs  be  matched  against  published 
claims,  but  they  could  be  matched  against 
each  other  to  determine  which  designs  were 
most  efficient.  This  was  a  task  that  the 
modeling  effort  performed  admirably.  A 
definitive  measure  of  the  throughput  of  the 
modeled  designs  was  provided  to  the 
Source  Selection  Board  (SSB). 
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Problems 


Of  the  three  instances  where  this 
information  was  provided  to  the  SSB,  two 
of  the  proposal  claims  were  upheld,  but  the 
third  fell  short.  This  apparent  shortfall 
forced  a  closer  look  at  the  text  of  the 
proposal.  Had  the  model  not  been 
constructed,  this  reexamination  may  have 
never  occurred. 

Acceptability 

The  modeling  effort  also  revealed 
another  problem  that  occurs  when 
attempting  to  evaluate  a  proposal.  During 
the  attempt  to  make  a  model  of  a  system 
that  is  described  on  paper,  it  is  not  rare  to 
come  across  inconsistencies  or  omissions 
that  may  escape  attention  if  one  relies  solely 
on  the  information  in  written  form.  This  is 
even  more  likely  to  occur  when  a  series  of 
competing  proposals  is  being  evaluated. 

During  the  course  of  the  effort 
presented  here  such  a  scenario  indeed 
happened.  The  definition  of  proposal  #4 
included  many  such  inconsistencies,  so 
many  that  this  proposal  was  removed  from 
consideration.  The  inability  to  construct  a 
model  of  this  proposal  revealed  an  immature 
design  that  was  quickly  discarded  and 
valuable  time  was  not  spent  deciding 
whether  or  not  this  design  was  feasible. 

Pioneering 

Lastly,  the  lessons  learned  when 
attempting  to  do  something  for  the  first  time 
are  invaluable.  The  trials  and  tribulations  of 
trailblazing  ofren  make  the  path  much 
smoother  for  those  that  follow.  Iteration 
spawns  improvement. 


Several  problems  prevented  us  from 
completely  reaching  the  objective  of  the 
modeling  support;  a  quantified  technical 
comparison  of  all  proposals,  from  being 
achieved.  They  were: 


0) 

Inadequate  time  for  modeling. 

(2) 

Incorrect  assumptions 

during 

modeling  preparation. 

(3) 

Inconsistent  completion 

of  the 

Data  Information  Forms. 

Time 

By  the  time  the  proposal  evaluation 
began,  the  time  allocated  to  modeling  had 
been  reduced  to  seven  days.  It  wasn’t 
enough  time. 

To  add  to  the  problem  the  CACD 
anticipated  modeling  no  more  than  3-5 
proposals  of  known  architecture  and 
protocol.  Additionally,  engineering  support 
by  TET  members  had  been  guaranteed. 
Because  such  a  low  level  of  effort  was 
anticipated,  with  readily  available 
engineering  support,  only  one  software 
engineer  was  allocated  to  the  project.  This 
level  of  support  was  insufficient. 

When  the  proposals  were  delivered 
to  the  CACD,  eleven  proposals  needed  to 
be  modeled.  Unexpectedly,  both  new 
architectures  and  protocols  had  been 
introduced.  In  particular,  the  introduction 
of  the  SONET  protocol  in  several  proposals 
took  everyone  by  surprise.  The  CACD  was 
not  prepared  for  the  magnitude  of  effort 
required  nor  for  the  technical  challenge  in 
understanding  and  modeling  a  new  protocol 
in  so  short  a  time.  Adding  to  the  problem 
was  the  inaccessibility  of  1 ET  engineers  to 
provide  technical  support.  They  were 
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Data  Information  Form 


located  across  town  in  a  room  with  no 
telephone.  Several  hours  were  required  to 
establish  communication  with  them  and  then 
they  were  often  busy  responding  to  other 
time  critical  issues. 

Assumptions 

The  program  director,  his  staff,  and 
the  CACD  anticipated  two  to  three  offerors 
with  proposals  which  were  well  understood 
using  architectures  and  protocols  which  had 
already  been  modeled.  Some  architectures 
were  so  well  understood  that  the  team 
believed  that  some  proposals  would  not 
need  to  be  modeled.  Finally,  the  experience 
during  the  “mock”  evaluation  had  confirmed 
the  team’s  confidence  in  the  adopted  MOE. 

All  of  these  assumptions  proved  to 
be  incorrect.  First,  there  were  five  offerors 
with  a  total  of  eleven  proposals.  Both  new 
architectures  and  protocols  were 
introduced.  No  one  in  the  CACD  had  any 
experience  with  the  SONET  protocol  and 
the  few  experienced  engineers  proved  to  be 
inaccessible.  Two  of  the  proposals  used 
architectures  which  invalidated  the  MOE 
thereby  reducing  the  value  of  modeling,  (ft/ 
HE,  #3  HE)  One  of  the  proposals  proved 
that  the  selected  MOE  was  not  the  only 
indicator  of  performance  that  should  have 
been  examined.  (U5)  Four  of  the  designs 
were  not  clearly  or  completely  defined 
enough  to  construct  a  model.  (fl2  HE,  U2 
LE,  U4  LE  U4  HE)  Fortunately,  two 
proposals  were  so  clearly  defined  and 
described  that  the  path  from  Data 
Information  Form  to  inclusion  in  the  CACD 
modeling  environment  was  very  clearly  laid 
out.  (#3,  U5)  (This  was  the  intent  of  the 
Data  Information  Form.)  This  left  three 
proposals  which  were  completely  modeled. 


Offerors  were  inconsistent  in 
completing  this  form  despite  the  RFP’s 
announcement  that  each  proposal  would  be 
modeled  in  OPNET™  and  its  requirement  to 
submit  a  completed  form  as  part  of  each 
proposal.  Some  offerors’  forms  reflected  an 
appreciation  of  the  OPNET“  tool  and  were 
so  well  completed  that  they  allowed  direct 
keying  of  the  model.  Others  were  so 
incomplete  and  contradictory  that  they 
confused  both  software  and  communications 
engineers.  In  hindsight,  a  correctly 
completed  form  should  have  been  included 
in  the  RFP  as  an  example  and  the 
importance  of  completing  the  form  should 
have  been  more  strongly  emphasized. 

Conclusion 

The  field  of  Modeling  and 
Simulation  has  advanced  significantly  in  the 
last  few  years.  Its  recent  growth  has  enabled 
groups  of  all  types  to  use  it  in  new  and 
interesting  ways.  One  of  those  ways  is  the 
innovative  use  of  simulations  to  evaluate 
bus  alternatives  during  the  proposal 
evaluation  phase  that  the  FODB  team  chose 
to  use.  This  was  the  first  time  that  a  Source 
Selection  Board  was  able  to  view  the  results 
of  models  based  on  the  designs  under  their 
evaluation. 

Several  lessons  were  learned  about 
the  use  of  modeling  and  simulation  for  a 
project  of  this  type  and  the  ways  in  which  it 
might  be  improved.  The  problems  described 
above  are  examples  of  the  lessons  learned 
with  this  project.  A  few  key  items  need  to 
be  changed  to  make  the  second  time  around 
a  complete  success.  More  time  has  to  be 
allocated.  Unfortunately,  time  is  one  thing 
that  source  selection  boards  may  not  enjoy. 
Given  this  constraint,  more  attention  must 
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be  focused  on  preparation  and  enforcement 
of  the  Data  Information  Forms. 

Better  preparation  would  not  allow 
show-stoppers  such  as  SONET  to  appear 
unexpectedly  during  the  evaluation  phase. 
Prior  knowledge  of  complex  protocols 
allow  the  bulk  of  the  simulation  work  to  be 
done  before  the  evaluation  phase  and 
outside  the  demanding  time  constraints. 

Enforcement  of  the  Data 
Information  Forms  would  eliminate  the 
ambiguity  and  contradiction  that  was 
apparent  in  forms  for  this  effort.  A  detailed 
example  form  must  be  included  in  the  RFP 
and  the  RFP  must  clearly  state  that 
incorrectly  completed  or  incomplete  data 
forms  will  disqualify  the  proposal. 

While  the  CACD’s  modeling 
support  failed  to  provide  a  complete 
quantifiable  comparison  of  all  proposals,  it 
did  contribute  to  the  elimination  of  some  of 
the  proposals.  It  also  demonstrated  the 
potential  of  modeling  and  simulation  for 
future  Source  Selection  Boards. 
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ABSTRACT 

The  Federal  government  has  a  history  of 
controlling  new  items  of  technology. 
Typewriters  and  facsimile  machines 
were  highly  regulated  when  first 
introduced  (and  still  are  to  some  extent) 
into  the  government  work  force.  The 
current  technology  under  rigid  control 
(although  it  is  certainly  not  new 
anymore)  is  the  computer  system  or 
information  resource.  Oingress  initiated 
control  of  this  area  of  technology  in 
1965^,  by  requiring  the  General  Services 
Administradon  (GS  A)  to  buy  or  delegate 
the  authority  to  buy  all  of  the  computers 
for  the  Federal  government.  At  that 
time,  the  perception  of  congress  was  that 
computers  were  very  expensive  toys  and 
that  the  individual  Federal  agencies  and 
their  personnel  could  not  be  trusted  to 
buy  or  manage  them.  The  term 
computer  is  being  used  generically  in 
this  paper  and  represents  every  level 
from  personal  (desk  top,  single-user)  to 
mainframe  (multi-user).  The  perception 
that  the  user  cannot  be  trusted  is  still 
widely  held.  This  is  evidenced  by  the 
regulations  established  by  GSA^  to 
control  information  resources.  This  is  in 
addition  to  the  layers  of  policy  and 
regulation  developed  by  each  of  the 
individual  agencies  and  the  services 
within  each  agency.  In  order  to  provide 
more  productive  and  efficient 
management  of  these  resources  the 
redundant  regulations  and  policies  need 
to  be  reduced.  In  addition,  management 
emphasis  needs  to  be  placed  on  the 


software  applications,  not  the  hardware 
platform. 

INTRODUCTION 

The  technology  of  computers  or 
information  systems  is  highly  dynamic. 
In  the  last  Hve  years  alone,  the 
technology  advances  in  computing 
hardware  have  decreased  the  cost  and 
footprint  (amount  of  space  taken  up  by 
the  hardware  platform)  of  computers 
dramatically  while  increasing  the  power 
and  capability  to  the  point  where  a 
desktop  computer  can  now  process  the 
same  information  that  a  mainframe  once 
did^.  One  example  of  this  technological 
advancement  is  Digital  Equipment 
Corporation's  VAX  8600  mini  compute. 
A  VAX  8600  bought  in  1986  at  a  cost  of 
$250,000.00  is  today  being  replaced  by 
the  VAXStation  3000  or  4()00  series 
desktop  workstations  at  a  cost  of 
between  $25,000.00  and  $50,000.00 
(depending  on  the  capability  and 
peripherals).  Today's  VAXStations  have 
a  computer  power  sixteen  times  greater 
than  the  1986  vintage  VAX  8600^. 
Software  costs  associated  with  this 
particular  manufacturer’s  computers  is 
comparative  to  the  hardware  cost.  This 
means  that  the  associated  purchase  cost 
for  the  software  operating  system  and 
applications  has  decrease.  In  those 
cases  when  one  computer  system  is 
traded  up  for  another  newer  technology 
computer,  the  software  would  just  be 
transferred  from  one  machine  to  another. 
The  only  associated  software  costs 
would  be  those  to  upgrade  from  one  type 
of  computer  to  another.  These  costs  are 
generally  minimal.  Conversely,  because 
of  the  capabilities  allowed  through 
available  software  appUcations,  software 
costs  and  the  proliferation  of  new 
applications  have  become  the  driving 
factor  when  determining  the  solution  to 
computing  requirements.  This  paper 
describes  an  approach  and  suggests  a 
solution  to  the  current  dilemma  that  the 
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government  finds  itself  in,  of  over 
regulation  and  auditing  requirements 
with  seemingly  no  obvious 
understanding  of  the  technology  and 
how  it  is  applied. 

DEVELOPMENT 

A  primary  factor  contributing  to  this 
phenomenon  of  over-control  at  the 
wrong  level  is  the  perceived  inability  to 
trust  the  user  (or  customer)  to  control 
costs  on  the  computing  systems  used  in 
the  workplace.  This  over  control  also 
implies  that  the  user  does  not  understand 
the  tool  on  his  desktop.  This  is  an 
incorrect  assumption.  Today's  computer 
users  have  become  so  comfortable  (in 
general)  with  the  desktop  computer  that 
it  is  considered  little  more  than  another 
administrative  tool.  In  today's 
technology,  the  government's  emphasis 
is  being  placed  on  the  wrong  component 
of  a  computer  system,  either  the 
hardware  platform  or  the  total  system 
(hardware  plus  software).  A  more 
practical  approach  to  meeting  the  users' 
requirements  while  also  controlling  costs 
is  some  level  of  software 
standardization.  This  approach  looks  at 
the  software  and  considers  the  hardware 
nothing  more  than  an  enabling  tool. 
This  also  supports  the  need  for  increased 
standardization,  linking  and  integration 
of  the  different  applications^. 

The  Federal  government  uses  two  types 
of  software,  commercially  developed 
and  available  and  custom  developed  for 
specific  applications  or  functions. 
Although  more  and  more  commercially 
developed  and  produced  software  is 
being  used  within  the  government  we 
still  have  a  tendency  to  custom  develop 
more  than  is  appropriate  or  necessary. 
Even  when  we  custom  develop,  we  do 
not  tend  to  look  at  other  agencies  or  sites 
that  have  already  had  software 
developed  that  could  be  modified  for 
other  uses.  Additionally,  we  are  still 


very  poor  at  requiring  the  proper 
documentation  and  rights  to  source  code. 
Software  designed  and  developed  for 
specific  Federal  (including  Defense) 
applications  have  had  much  attention 
paid  to  them.  Therefore,  there  are  a 
myriad  of  specifications,  standards  and 
processes  written  and  applied  to  that 
issue.  That  aspect  of  software  control 
requires  separate  attention  and  will  not 
be  dealt  with  in  this  paper. 

There  has  not  been  a  proportionate 
degree  of  attention  applied  to  the  use  of 
commercially  developed  and  Federally 
used  software.  In  the  commercial 
software  arena,  we  (the  government) 
have  failed  to  standardize  on  even  basic 
administrative  software,  thus  incurring 
unnecessary  costs  by  acquiring  multiple 
software  application  programs  that 
accomplish  the  same  purpose  but  are 
needed  to  read  or  write  to  others  in  our 
community  (co-workers  or  associates  at 
other  sites). 

Use  of  commercially  available  software 
instead  of  custom  developed  software  is 
strongly  promoted  within  the  Federal 
government.  One  of  several  reasons  for 
this  is  that  many  of  the  functional  needs 
of  the  Federal  government  personnel  are 
general  purpose  in  nature.  These  general 
purpose  functions  are  the  same  as  that 
for  any  other  organization,  business  or 
personal  requirement.  As  a  result,  there 
are  many  applications  written  for  any 
type  of  business  or  personal  function  that 
can  be  hosted  by  multiple  operating 
systems  and  any  level  of  computer 
system  (single  to  multi-user)  in  the 
commercial  market  place.  Therefore,  the 
application  development  costs  have 
already  been  borne  by  a  commercial 
developer.  As  a  result  of  the  fact  that 
this  developer  has  a  very  wide  customer 
base  and  his  product  is  competing  with  x 
quantities  of  other  companies  similar 
application  products,  the  cost  is 
comparable  to  all  other  similar  programs 
or  products  on  the  market.  It  is 
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generally  more  cost  effective  to  buy  a 
commercially  developed  business 
program  product  than  custom  design  and 
develop  a  program  for  Federal 
application.  This  would  support  the 
Federal  government's  position  on  using 
commercial  applications  over  custom 
developed  applications.  Unfortunately, 
the  use  of  the  commercial  applications 
has  had  no  standardization  applied.  The 
high  costs  applied  to  use  of  computers  in 
the  Federal  government  are  in  part  a 
result  of  this. 

The  Federal  government  needs  to 
promote  and  support  a  reasonable 
approach  or  set  of  standard  parameters  in 
the  use  of  commercial  applications. 
Currently,  there  are  no  standards  within 
the  government  for  use  of  software 
application  pro^ams.  This  problem 
ripples  out  from  incompatibility  within  a 
site  or  facility  to  organizations  nation 
wide.  This  lack  of  standardization  has 
resulted  in  a  need  for  multiple  software 
programs  to  support  an  individual  user's 
functional  needs.  An  example,  word 
processing,  has  many  program 
applications  written  for  it  and  a  user 
needs  multiple  companies  copies  in 
order  to  read  and  write  to  electronically 
transmitted  files  or  documents.  The 
result  of  this  need  to  host  multiple 
manufacturers  software  applications  for 
each  function  are  excessive  and  runaway 
costs.  This  is  an  unnecessarily  wasteful 
use  of  Federal  funds.  This  also  supports 
Congress'  perception  that  the  individual 
agencies  and  customers  cannot  be  trusted 
to  control  costs. 

CONCLUSION 

Federal  control  (standardization)  for 
commercially  available  software 
application  programs  at  the  format  level 
would  allow  individual  users  the  ability 
to  use  the  computing  technology  tool  of 
their  choice  while  using  a  standardized 
and  cost  effective  approach  to  the 
acquisition  of  the  software  programs. 


The  Federal  acquisition  and  use  method 
would  then  be  more  consistent  with 
prevailing  commercial  business 
practices.  The  current  Federal 
acquisition  process  could  more  easily 
procure  needed  software  using  the 
preferred  competitive  method.  The 
control  would  then  be  at  the  more 
practical  application  or  functional  level. 
I  believe  that  this  simplified  approach  to 
control  would  result  in  a  dr^atic  and 
immediate  downturn  of  costs  applied  to 
our  Federal  information  resources 
(computers).  I  believe  that  this  approach 
would  also  lessen  the  requirement  for 
regulations  and  audit,  thereby  reducing 
associated  administrative  and  labor 
costs. 


f  Brooks  Act  of  1965  Public  Law  89- 
(10/65) 

^Federal  Information  Resources 
Management  Regulation  ( 1 2/90) 

^Jonathan  Weber, LPS-AnsskS 
Times. ,  "Computer  Giants  Drag  Out 
Their  Big  Guns  in  Workstation 
Wars"  10  November  1992  sec.  D  1-2 
'^Phone  conversation  with  Bill  Kert, 
Ridgecrest  Digital  Equipment 
Corporation  Branch  manager  on  15 
October  1992. 

Spatapro.  "The  Current  Status  of 
Technology"  April  1992  sec.  1012  1- 
12 
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An  issue  of  great  concern  in  public  administra¬ 
tion  has  been  whether  the  Federal  workforce  is 
qualified  to  carry  out  the  many  tasks  assigned  to 
it.  Research  conducted  by  the  U.S.  Merit  Sys¬ 
tems  Protection  Board  attempts  to  answer  this 
question  for  the  more  than  30,000  people  em¬ 
ployed  as  Federal  contract  specialists. 

This  research  is  based  on  surveys  of  current 
contract  specialists,  their  immediate  supervi¬ 
sors,  members  of  the  Senior  Executive  Service, 
and  private  contractors.  Based  on  these  surveys, 
it  was  found  that  although  both  incumbents  and 
their  supervisors  were  positive  about  the  capa¬ 
bilities  of  current  contract  specialists,  addi¬ 
tional  training  is  needed.  Responses  from  the 
two  client  groups  indicated  that  although  mem¬ 
bers  of  the  procurement  workforce  were  capable 
in  some  ways,  procurement  actions  take  too  long 
and  the  procurement  process  does  not  always 
serve  the  best  interests  of  the  Government. 

Responses  from  all  four  groups  attribute  many 
of  the  problems  to  an  increase  in  the  complexity 
of  the  procurement  process  during  the  last  de¬ 
cade.  According  to  respondents,  the  process  has 
become  so  complicated  that  it  is  often  dijficult 
for  the  average  contract  specialist  to  meet  the 
needs  of  his!  her  organization  in  a  timely  manner 
while  ensuring  that  no  reguiations  are  broken. 

Introduction 

In  recent  years,  members  of  Congress  and  the 
public  administration  community  have  raised 
concents  about  a  possible  decline  in  the  quality  of 
the  Federal  workforce  and  the  effect  it  may  have 
on  Government  operations.  Many  have  sug¬ 
gested  that  the  refusal  to  grant  pay  increases 
needed  to  maintain  comparability  with  the  pri¬ 
vate  sector  over  the  past  decade  and  a  half  and  the 
deteriorating  image  of  the  Federal  workforce 
have  worked  together  to  reduce  the  number  of 
highly  qualified  persons  employed  in  the  Federal 
workforce.  As  noted  in  several  reports  by  the 
U.S.  Merit  Systems  Protection  Board  (MSPB), 
there  is  also  a  perception  among  many  Federal 
managers  that  tliere  has  been  a  marked  decline  in 
the  quality  of  new  hires. 

In  response  to  these  concerns,  both  MSPB  and 
the  U.S.  Office  of  Personnel  Management  (0PM) 
set  out  to  look  at  the  issue  of  workforce  quality. 
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Since  the  Federal  workforce  is  composed  of  over 
2,000,000  employees  (excluding  the  postal  ser¬ 
vice)  working  in  hundieds  of  different  jobs,  it  was 
immediately  obvious  that  it  would  be  impossible 
if  not  meaningless  to  look  at  the  quality  of  the 
entire  Federal  workforce  as  a  whole.  Instead 
MSPB  and  0PM  decided  that  it  would  be  more 
effective  if  each  agency  conducted  separate  stud¬ 
ies  looking  at  the  quality  of  employees  in  selected 
occupations.  The  occupation  chosen  by  MSPB 
was  contracting  specialists,  i.e.,  persons  working 
in  the  GS-1 102  occupational  series. 

Why  study  employees  working  in  the  GS-1 102 
series?  Simply  put.  there  may  be  no  area  where 
there  has  been  greater  concern  about  the  quality 
of  Federal  workers  and  tlie  work  they  perform 
than  in  the  procurement  of  goods  and  services 
from  the  private  sector.  During  the  1970’s  and 
early  1980's  there  were  several  highly  publicized 
incidents  which  involved  questionable  Federal 
spending.  Employees  responsible  for  Govern¬ 
ment  procurements  were  severely  criticized  for 
spending  too  much  money  to  purchase  a  variety 
of  products  ranging  from  coffee  pots  to  major 
weapon  systems. 

There  were  two  additional  reasons  for  choosing 
to  look  at  this  segment  of  the  Federal  workforce. 
The  first  was  the  large  number  of  people  em¬ 
ployed  in  this  profession  (over  31.000  in  1991) 
and  the  second  was  vast  sums  spent  by  contracting 
specialists  to  purchase  the  gooils  and  services 
needed  by  the  Government  (contract  specialists 
were  responsible  for  spending  over  191  billion 
dollars  in  1990).  Given  the  potential  impact  that 
this  portion  of  the  Feileral  workforce  on  Govern¬ 
ment  operations,  even  relatively  modest  changes 
in  the  quality  of  the  contract  specialist  workforce 
could  have  tremendous  effects  on  the  quality  of 
seiwices  delivered  by  the  Federal  Government  to 
the  American  public. 

The  primary  puipose  of  this  study  was  to  deter¬ 
mine  the  quality  of  both  the  Government’s 
contracting  specialist  employees  and  the  work 
that  they  perfonn.  Since  it  is  difficult  to  develop 
an  adequate  single  definition  of  what  is  meant  by 
quality  in  the  area  of  Federal  procurement,  this 
study  tried  to  assess  t]iiality  from  four  dilTerent 
perspectives.  Two  nf  these  perspectives  were 
provided  by  people  currently  w  nvkine  in  13-ileral 
procurement — currentcontractingspeeialistsaiul 


their  supervisors.  The  other  two  perspectives  sors.  These  surveys  asked  employees  how  well 

came  from  clients  for  the  services  provided  by  they  performed  the  aspects  of  theirjobs  that  were 

contract  specialists — members  of  the  Senior  identified  as  critical  to  successful  performance 

Executive  Service  and  private  sector  vendors  through  traditionaljob  analysis  techniques.  They 

who  provide  goods  or  services  to  the  Govern-  were  also  asked  to  tell  us  about  their  back- 

ment  under  contracts  administered  by  contract  grounds  and  the  conditions  under  which  they 

specialists.  By  integrating  the  perspectives  of  worked.  In  order  to  get  a  second  perspective  on 

employees,  their  supervisors,  senior  Govern-  each  employee’s  performance,  we  distributed  to 
ment  managers,  and  private  contractors,  we  at-  each  employee’s  supervisor  a  survey  very  similar 

tempted  to  present  a  complete  picture  of  the  to  the  one  sent  to  contract  specialists.  The 

extent  to  which  an  important  segment  of  the  questionnaire  for  supervisors  differed  from  the 

Federal  workforce  is  accomplishing  its  assigned  one  sent  to  employees  in  that  it  asked  supervisors 

tasks.  to  provide  a  confidential  assessment  of  their 

subordinates’  performance.  Information 
The  infomiation  presented  in  this  paper  was  concerning  the  quality  of  service  provided  by 

collected  through  suiweys  distributed  to  each  of  contract  specialists  was  obtained  from  surveys 

the  four  different  groups  just  mentioned.  Sur-  senttoabout2700membersoftheSeniorExecu- 

veys  were  sent  to  approximately  9,300  cunent  tive  Service  and  900  private  contractors, 

contract  specialists  and  their  immediate  supeiwi- 


Table  1 

Average  Self  and  Supervisory  Ratings  of  Performance  on  Critical  Tasks 

by  GS-1102  Employees 

(Ratings  Provided  for  Employees  at  Grades  GS-9  and  Above) 

Ave. 

Ave.  Self 

Supervisory 

Task 

Rating 

Rating 

Conduct  market  research 

3.91 

3.70 

Review  statements  of  work 

3.88 

3.77 

Detennine  competition  requirements 

4.02 

3.89 

Identify  price-related  factors 

3.63 

3.59 

Review  business  management  factors 

3.73 

3.63 

Identify  contract  type 

3.93 

3.78 

Prepare  requests  for  proposals 

4.14 

3.92 

Determine  low  bids 

4.02 

3.89 

Review  technical  evaluations 

3.91 

3.75 

Evaluate  offers 

3.95 

3.76 

Conduct  co.st  analysis 

3.91 

3.72 

C>ctermine  competitive  range 

3.91 

3.74 

Develop  negotiation  strategy 

3.87 

3.68 

Conduct  negotiations 

3.90 

3.75 

Determine  standards  of  responsibility 

3.98 

3.84 

Respond  to  protests 

3.63 

3.57 

Monitor  compliance 

3.81 

3.71 

Identify  contractual  remedies 

3.67 

3.56 

Determine  contract  modifications 

4.05 

3.84 

Research  claims 

3.60 

3.57 

Average  for  20  tasks 

3.87 

3.73 

Views  From  the  Procurement  Workforce 

In  completing  our  survey,  employees  were  asked 
to  rate  their  own  performance  using  a  five-point 
scale  on  20  job  tasks  that  had  been  identified  as 
critical  to  the  work  performed  by  contract  spe¬ 
cialists  by  the  Federal  Acquisition  Institute. 
Employee  and  supervisory  ratings  on  each  of  the 
20  tasks  are  shown  in  table  1.  A  response  of  “1” 
meant  that  they  believed  they  were  unable  to 
perform  the  task  and  a  “5”  meant  that  they 
perfomied  this  task  exceptionally  well.  A  “3” 
meant  that  they  performed  this  task  at  an  accept¬ 
able  or  average  level. 

Comparisons  of  the  perfonnance  ratings  pro¬ 
vided  by  both  employees  and  their  supei'visors 
showed  that  both  groups  were  quite  positive 
about  the  quality  of  the  work  performed  by 
members  of  the  contract  specialist  workforce  on 
these  20  critical  tasks.  In  fact,  the  overall  mean 
rating  of  3.87  indicates  that  the  overwhelming 
majority  of  employees  (on  average,  65  percent  of 
the  employees)  thought  that  they  perfonned 
these  tasks  in  an  above  average  manner.  Super¬ 
visors  also  rated  their  employees  positively  but 
slightly  lower,  with  an  average  rating  of  3.73 
(reflecting  that,  on  average,  58  percent  of  the 
supervisors  reported  that  their  subordinates  per¬ 
fonned  tliese  tasks  in  an  above  average  manner). 


To  the  extent  that  self  and  supervisory  ratings  of 
performance  on  tliese  20  critical  tasks  are  indica¬ 
tive  of  quality,  it  would  appear  that  tlie  quality  of 
work  performed  by  members  of  the  GS-1102 
workforce  is  quite  high.  Even  so,  apparently  a 
relatively  large  percentage  of  both  employees 
and  supervisors  believed  that  there  is  substantial 
room  for  improvement  since,  on  average,  one- 
third  of  both  the  employees  and  tlieir  supervisors 
said  that  contract  specialists  need  at  least  a 
moderate  amount  of  additional  training  with 
regard  to  each  of  these  tasks. 

The  fact  that  so  many  employees  and  their  super¬ 
visors  claimed  that  additional  training  was  needed 
would  appear  to  contradict  the  fact  the  vast 
majority  of  the  employees  (on  average,  96  per¬ 
cent)  said  that  they  were  peifomiing  at  least 
adequately.  This  apparent  discrepancy  was  at 
least  partially  explained  in  written  comments 
received  from  numerous  employees  in  response 
to  the  survey.  According  to  our  respondents, 
tliey  simply  cannot  keep  up  with  the  changes 
occurring  in  both  the  Federal  Acquisition  Regu¬ 
lations  (FAR)  ajid  agency  implementation  of 
these  regulations.  As  a  result,  training  is 
continually  needed  in  order  to  keep  cuirent  with 
the  ever-changing  Federal  contracting 
environment. 


Table  2 

Self  and  Supervisory  Ratings  of  Abilities  Essential  to  the  Work  of  GS-1102  Employees 
(Percent  of  GS-9  and  Above  Employees  Rated  "Adequate"  and  "Outstanding") 

Self  Ratings  Supervisory  Ratings 


Ability 

Adequate 

Outstanding 

Adequate 

Outstanding 

Directing  Work  Activities 

50 

41 

54 

28 

Planning  aixi  Organizing 

42 

52 

50 

37 

Human  Relations  Ability 

46 

46 

52 

32 

Analytical  Ability 

47 

.45 

48 

37 

Oral  Communications 

47 

46 

52 

37 

Writing  Ability 

46 

46 

51 

33 

Ability  to  Innovate 

54 

32 

53 

22 

Ability  to  Initiate  Action 

48 

45 

50 

34 

Conducting  Negotiations 

48 

36 

49 

33 

Average 

48 

43 

51 

33 
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In  addition  to  collecting  infonnation  concerning 
peiformance  of  work-related  tasks,  our  survey 
asked  employees  to  rate  themselves  on  nine  basic 
abilities  that  were  identified  as  essential  to  per- 
fonning  the  contract  specialist  duties,  again  as 
identified  by  job  analyses  conducted  by  the  Fed¬ 
eral  Acquisition  Institute.  These  abilities  are 
listed  in  table  2. 

As  shown  in  this  table,  on  average,  48  percent  of 
the  eiTiployees  rated  themselves  adequate  in  tenns 
of  these  abilities,  while  43  percent  rated  them¬ 
selves  outstanding.  By  comparison,  supervisors 
rated  their  subordinates  as  adequate  51  percent 
of  the  time  and  outstanding  33  percent  of  the 
time.  Nevertheless,  once  again,  the  results  indi¬ 
cated  that  there  is  substantial  room  for  improve¬ 
ment  with  regard  toeach  of  the  nine  abilities  since 
many  employees  and  their  supervisors  believed 
that  additional  training  is  needed  in  each  of  these 
areas.  On  average,  supervisors  reported  that  55 
percent  of  their  subordinates  need  additional 
training  in  tenns  of  these  nine  abilities.  Both 
employees  and  supervisors  agreed  that  the  area 
with  the  most  room  for  improvement  was 
conducting  negotiations.  For  their  part,  supervi¬ 
sors  also  thought  that  significant  improvements 
could  be  made  through  training  in  analytical 
ability  and  the  ability  to  write. 

Another  important  measure  of  the  quality  of  an 
employee’s  work  is  the  official  performance  ap¬ 
praisal  rating  that  a  supervisor  is  required  to  give 
each  subordinate  every  year.  Until  recently,  this 
rating  was  nonnally  provided  using  a  five-point 
scale,  running  from  “unsatisfactory”  (or  “1”)  to 
“outstanding”  (or  “5”).  Employees  responding 
to  our  survey  were  asked  what  rating  they  had 
received  during  their  most  recent  fonnal  perfor¬ 
mance  appraisal.  In  response  to  this  question,  31 
percent  said  that  they  were  rated  “outstanding” 
and  45  percent  “exceeds  fully  successful.”  Only 
24  percent  indicated  that  they  had  been  rated 
“fully successful.”  Fewerthan  1  percentsaid that 
they  had  been  rated  “less  than  fully  successful.” 
Similardistributions  of  ratings  were  reported  for 
each  of  the  previous  2  years  as  well. 

Unfortunately,  the  formal  peiformance  rating 
given  to  an  employee  each  yeiu'can  be  influenced 
by  a  number  of  factors  other  than  the  employee’s 
work  perfonoance.  In  fact,  research  by  OPM  has 
shown  that  when  supervisors  were  asked  to 
provide  confidential  ratings  on  the  peiformance 
of  employees  they  supervised,  these  ratings  were 
frequently  significantly  different  from  tlie  official 


peiformance  rating  they  gave  the  employee.  For 
this  reason,  we  decided  to  ask  supervisors  to 
provide  confidential  ratings  of  their  subordinate’s 
performance.  As  was  done  when  providing  an 
employee  witli  an  official  performance  rating, 
supervisors  were  asked  to  rate  employees  on  a 
five-point  scale,  with  a  “1”  being  assigned  to  the 
lowest  rating  and  a  “5”  to  tlie  highest.  Figure  1 
shows  the  distribution  of  both  the  most  recent 
official  ratings  reported  by  employees  and  the 
average  overall  research  rating  provided  by  su¬ 
pervisors  for  their  subordinates. 

As  this  chart  shows,  supervisors  frequently  gave 
lower  ratings  to  their  subordinates  when  asked  to 
provide  confidential  ratings  of  employee  perfor¬ 
mance  than  employees  reported  were  given  to 
them  in  conjunction  with  the  official  performance 
appraisal  process.  Nevertheless,  the  research 
ratings  we  obtained  are  still  fairly  high.  In  fact, 
over  59  percent  of  the  employees  received  aver¬ 
age  research  ratings  that  placed  them  in  a  cat- 
egoi7  above  “fully  successful.”  Clearly,  in  terms 
of  both  the  formal  performance  appraisal  ratings 
and  the  confidential  research  ratings,  supervisors 
believed  that  the  contract  specialists  who  work 
for  them  generally  do  a  good  job. 

In  addition  to  analyzing  ratings  on  abilities,  per¬ 
formance  of  job-related  tasks,  and  overall  perfor¬ 
mance,  we  collected  information  on  a  number  of 
additional  factors  that  could  be  related  to  the 
quality  of  work  performed  by  Federal  contract 
specialists.  Tliis  included  information  concerning 
the  fonnal  educational  backgiounds  of  contracting 
specialists.  Tlie  issue  of  education  is  one  of 
particular  concern  among  senior  management 
officials  in  procurement.  In  discussions  witli 
the.se  officials,  it  was  suggested  that  too  few  of 
the  people  entering  the  GS-1102  series  during 
the  last  10  to  15  years  had  completed  sufficient 
levels  of  education  to  prepare  them  to  perform 
their  work. 

Given  the  concerns  expres.sed  to  us  by  senior 
officials  in  procurement,  it  was  somewhat  sur¬ 
prising  to  find  that  over  half  (56  percent)  of  the 
people  working  in  this  area  have  earned  at  least 
a  bachelor’s  degree  and  about  one  in  six  has 
obtained  at  least  a  master’s  degree.  There  was 
some  evidence,  however,  to  support  the 
contention  of  many  senior  procurement  manag¬ 
ers  that  persons  entering  the  GS-1102  series 
during  the  last  10  to  15  years  were  somewhat  less 
likely  to  have  achieved  the  educational  level 
attained  by  those  who  entered  the  contract  spe- 
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Figure  1 

Disribution  of  Official  and  Average  Research  Ratings  for 
GS-1102  Employees 
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cialist  workforce  before  that  time.  A  breakdown 
by  length  of  time  working  in  the  GS-1 102  series 
revealed  that  over  61  percent  of  the  persons 
working  in  this  series  for  15  or  more  years  had 
attained  at  least  a  bachelor’s  degree.  By 
comparison,  slightly  fewer  than  53  percent  of  the 
people  entering  this  series  during  the  last  1 5  years 
had  completed  the  same  level  of  education. 

Self  reports  of  grades  obtained  in  school  also 
suggest  that  employees  did  quite  well  in  the 
course  of  getting  their  degrees.  The  overall  grade 
point  average  (GPA)  reported  by  contract  spe¬ 
cialists  who  completed  college  was  3.13  on  a 
four-point  scale.  Moreover,  the  GPA  for  the 
employee’s  last  2  years  was  3.29  and  3.35  in  his 
or  her  major  field.  Additionally,  more  than  75 
percent  of  the  employees  reported  that  they  were 
in  the  top  25  percent  of  their  graduating  class. 

Based  on  the  results  just  presented,  it  is  clear  that 
in  tenns  of  the  direct  match  between  critical  job 
requirements  and  workforce  characteristics,  both 
supervisors  and  employees  believed  that  the  qual¬ 
ity  of  the  procurement  workforce  is  at  least 


adequate,  if  not  superior.  Even  so,  supervisors, 
in  particular,  reported  that  there  is  substantial 
room  for  improvement  as  indicated  by  their  belief 
that  significantly  more  training  is  needed  for  large 
portions  of  the  workforce.  Perhaps  even  more 
importantly,  comments  provided  by  both  em¬ 
ployees  and  supervisors  in  response  to  our  survey 
revealed  that  there  were  a  number  of  potentially 
substantial  problems  which  both  groups  believed 
limit  the  quality  of  the  work  performed  by  contract 
specialists. 

Literally  thousands  of  comments  were  received 
from  both  employees  and  supervisors,  many  of 
whom  repeated  the  same  themes.  One  recurrent 
theme  among  the  comments  provided  by  em¬ 
ployees  was  the  belief  that  contract  specialists  are 
being  overwhelmed  by  changes  to  the  procure¬ 
ment  process.  According  to  many  of  these 
contract  specialists,  this  situation  was  frequently 
made  even  more  difficult  since  either  adequate 
money  for  training  was  not  available  in  their 
organization  or  their  work  load  was  so  heavy  that 
they  were  not  permitted  to  take  the  time  to  attend 
training. 
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From  the  perspective  of  the  supervisors  who 
provided  written  comments,  the  main  problems 
facing  the  procurement  workforce  were  the  vol¬ 
ume  of  the  work  and  the  complexity  of  the 
procurement  process.  Of  course,  these  problems 
are  interrelated.  As  was  the  case  with  the  contract 
specialists  themselves,  these  supervisors  reported 
that  the  procurement  process  has  become  so 
complex  that  the  even  small  procurement  actions 
can  take  an  inordinate  amountof  time  andinvolve 
excessive  paperwork.  From  the  perspective  of 
these  supervisors,  there  are  so  many  rules  that 
contract  specialists  in  many  agencies  have  little  or 
no  discretion.  Unfortunately,  these  same  super¬ 
visors  thouJtht  that  this  sometimes  led  to  pro¬ 
curement  decisions  that  seemingly  made  little 
business  sense. 

Tile  increasing  complexity  of  the  procurement 
process  frequently  also  has  the  effect  of  lengthen¬ 
ing  the  procurement  process  and,  as  a  result, 
there  was  constant  pressure  from  management  to 
speed  up  a  process  that  has  been  slowed  down  by 
regulatory  requirements.  According  to  some 
employees,  this  sometimes  resulted  in  a  tendency 
for  management  to  put  tremendous  pressure  on 
contract  specialists  to  bend  the  rules. 

For  many  employees  the  situation  becomes  all 
but  intolerable  at  the  end  of  the  fiscal  year,  when 
organizations  are  concerned  with  spending  all  of 
the  money  allocated  to  them  so  that  they  do  not 
lose  this  money  or  have  their  budgets  cut  the 
following  year.  In  their  written  comments,  a 
number  of  respondents  told  us  that  their  organi¬ 
zations  were  more  interested  in  obligating  all  of 
the  money  they  were  budgeted  before  the  end  of 
each  year  than  they  were  in  making  sure  that 
procurements  are  made  in  confonnance  with 
applicable  rules  and  regulations. 

Although  contract  specialists  may  be  pressured 
to  complete  contracts  and  obligate  dollars  as 
quickly  as  possible,  their  agencies  are  legally 
responsible  for  the  integrity  of  the  contracts  they 
write.  Despite  pressures  of  the  sort  described  in 
their  written  comments,  apparently  most  em¬ 
ployees  recognized  that  breaking  the  rules  is  not 
an  acceptable  response.  In  fact,  discussions  with 
senior  procurement  officials  concerning  the  in- 
teipretation  of  survey  results  suggested  that  the 
emphasis  on  legal  and  defensible  contracts  may 
be  so  great  in  some  organizations  that  it  has 
become  the  overriding  concern  in  the  minds  of 
many  contract  specialists.  According  to  one 
supervisor:  “Playing  it  safe  and  by  the  book  is 
what  gets  rewarded.”  This  is  not  to  say  that 


supervisors  were  suggesting  that  employees 
should  break  the  rules,  only  that  the  emphasis  has 
shifted  from  finding  legal  ways  of  meeting  the 
needs  of  the  organization  to  being  sure  to  protect 
oneself. 

Although  it  is  certainly  important  for  a  contract 
to  be  able  to  stand  in  the  face  of  a  protest,  too 
much  emphasis  on  protecting  oneself  apparently 
fosters  a  closed-minded  approach  to  contracting. 
According  to  many  supervisory  comments,  this 
type  of  thinking  can  lead  to  an  authoritarian 
enforcementmentality  where  contract  specialists 
rigidly  apply  the  rules  that  they  have  learned, 
often  without  adequate  understanding  of  the 
options  available  to  them  and  sometimes  even 
without  a  sufficient  appreciation  of  the  missions 
of  the  organizations  they  support.  This  type  of 
tliinking  probably  also  contributes  to  tlie  belief 
held  by  many  supervisors  that  their  subordinates 
lack  the  ability  to  innovate  and  lack  creativity. 

Based  upon  both  survey  responses  and  written 
comments,  it  appears  that  while  supervisors  be¬ 
lieved  that  most  of  tlie  people  who  worked  for 
them  had  the  ability  to  perform  the  tasks  required 
of  them  under  the  best  of  conditions,  they  also 
thought  that  their  subordinates  seldom  possessed 
the  abilities  needed  to  devise  creative  legal  solu¬ 
tions  to  organizational  problems  given  the  situa¬ 
tions  in  which  they  typically  find  themselves.  As 
a  result  of  the  complexity  of  the  Federal  procure¬ 
ment  system  and  organizational  pressure  to  obli¬ 
gate  all  allocated  money,  contract  specialists 
were  frequently  faced  with  a  dilemma  of  how  to 
provide  the  service  required  by  their  organization 
without  breaking  the  rules.  U nfortunately ,  under 
these  conditions  it  appears  that  only  those  em¬ 
ployees  possessing  an  unusual  amount  of  analyti¬ 
cal  ability  and  creativity  were  able  to  successfully 
meet  both  of  these  goals.  According  to  supervi¬ 
sors,  employees  possessing  those  abilities  to  a 
lesser  extent  typically  either  chose  to  bend  the 
rules  or  developed  an  inflexible  approach  to 
contracting  that  may  alienate  managers  in  the 
organizations  in  which  they  work. 

Client  Views  of  Procurement  Workforce 
Quality 

In  addition  to  information  provided  by  persons 
working  in  procurement,  valuable  infonoation 
about  workforce  quality  can  be  collected  from 
the  clients  for  whom  goods  or  services  are  ob¬ 
tained.  For  this  reason,  a  third  and  founh  per¬ 
spective  of  procurement  work  quality  was  pro¬ 
vided  by  persons  who  were,  in  a  real  sense,  clients 
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for  the  work  performed  by  contract  specialists.  private  contractors  viewed  several  aspects  of  the 

One  facet  of  client  satisfaction  was  provided  work  perfomied  by  contract  specialists  quite 

through  information  collected  from  a  survey  of  positively, 

current  members  of  the  Senior  Executive  Ser¬ 
vice.  An  additional  perspective  was  obtained  Althoughseniorexecutivesweregenerallyposi- 

through  surveys  sent  to  private  sector  companies  tive  concerning  many  aspects  of  the  work  per- 

that  currently  have  contracts  with  the  Govern-  formed  by  contract  specialists,  they  did  indicate 

ment  and  whose  representatives  have  extensive  that  there  was  room  for  improvement  in  several 

dealings  with  Federal  contract  specialists.  Per-  areas.  For  example,  senior  executives  were 

sons  responding  to  each  of  these  surveys  were  somewhat  less  satisfied  about  the  extent  to  which 

asked  about  the  overall  effectiveness  of  Federal  people  working  in  procurement  maintained  a 

procurement  practices,  the  quality  of  products  positive  service  orientation  towards  clients  or 

provided  to  the  Government  by  the  private  sec-  colleagues  in  their  organization.  Similarly,  38 

tor,  and  the  quality  of  work  performed  by  the  percent  oftheseniorexecutives  said  that  with  the 

individual  Federal  contract  specialists  with  whom  possible  exception  of  some  routine  tasks,  work 

they  deal.  by  contract  specialists  supporting  their  organiza¬ 

tion  was  usually  not  completed  in  a  timely  man- 
In  analyzing  the  results  of  our  surveys  we  were  ner. 
somewhat  surprised  to  find  that  in  many  ways  the 

senior  executives  responding  to  our  survey  As  was  the  case  for  supervisors,  a  very  common 

thought  quite  highly  of  the  contract  specialists  theme  in  the  comments  provided  by  members  of 
who  supported  their  organizations  and,  as  shown  the  SES  was  that  too  many  contract  specialists 

in  table  3,  both  the  senior  executives  and  the  were  “rule-bound”  and  prone  to  attempt  to  apply 


Table  3 

SES  and  Contractor  Opinions  Concerning  GS-1102  Employees 

(Percent  of  SES  Members  and  Contractors  Responding  to  the  Question  "Federal 

contract  specialists  (are):") 


SES  Members  Contractors 


Item 

Agree 

Disagree 

Agree 

Disag 

Knowledgeable  about 

Federal  procurement 
procedures. 

90 

4 

68 

15 

Apply  Government 
regulations  accurately 
and  fairly. 

79 

8 

58 

20 

Well  traih^. 

74 

13 

63 

24 

Generally  helpful. 

74 

14 

66 

16 

Provide  prompt  responses 
to  questions. 

62 

21 

56 

29 

Notes:  1.  The  column  labelled  "Agree"  includes  both  "Strongly  Agree"  and  "Tend  to  Agree"  respinses.  Similarly, 
the  "Disagree"  column  includes  both  "Strongly  Disagree"  and  "Tend  to  Disagree"  responses,  2.  "Neither  Agree  nor 
Disagree"  and  "Don't  Know"  responses  were  omitted. 


rules  bliiuily  to  a  variety  of  contracting  situations 
without  adequately  understanding  the  objectives, 
goals,  or  mission  of  theorganization.  These  same 
senior  executives  tended  to  see  contract  special¬ 
ists  as  myopic  defenders  of  contract  regulations 
rather  than  as  team  players  whose  purpose  was  to 
get  the  job  done  effectively.  As  such,  contract 
specialists  were  seen  as  more  inclined  to  follow 
a  procurement  cookbook  than  to  look  for  inno¬ 
vative  and  legal  solutions  to  organizational  needs. 
This  perception  on  the  part  of  some  SES  mem¬ 
bers  was  also  reflected  in  the  responses  to  a 
question  about  the  creativity  of  the  work  per- 
fonned  by  contract  specialists. 

Approximately  39  percent  of  the  senior  execu¬ 
tive  respondents  said  that  the  contract  specialists 
supportingtheirorganization  showed  inadequate 
creativity  in  their  work. 

Although  critical  of  their  contract  specialists  in 
the  aforementioned  ways,  senior  executives  rec¬ 
ognized  that  some  of  the  concerns  they  raised 
may  have  resulted  from  the  conditions  under 
which  contract  specialists  frequently  must  work. 
Eorexample,3S  percent  of  the  SES  members  said 
that  contract  specialists  who  supported  their 
organization  were  too  often  asked  to  perform 
difficult  tasks  under  tight  time  constraints  (only 
20  percent  disagreed).  Atlditionally,  most  (51 
percent)  indicated  that  contract  specialists  were 
often  under  undue  pressure  as  a  result  of  having 
too  much  work  to  perform. 

As  was  shown  in  table  3,  private  contractors  also 
held  agenerally  positive  view' of  Federal  contract 
specialists  in  at  least  several  areas.  Like  the 
senior  executives,  contractors  felt  that  Federal 
contract  specialists  were  basically  knowledge¬ 
able,  well-trained,  and  helpful.  Even  so,  they 
tended  to  be  somewhat  less  positive  than  were 
the  senior  executives  with  respect  to  the  .same 
items. 

Responses  to  other  items  on  the  survey  revealed 
that  there  were  also  areas  of  significant  concern 
among  private  contractors.  Although  85  percent 
of  the  contractors  said  that  their  firm  has  a  good 
working  relationship  with  the  contract  .specialist 
assigned  to  their  contract,  almost  one-third  of 
these  same  private  sector  respondents  did  not 
think  that  contract  specialists  were  prepared  to 
discuss  substantive  issues  concerning  the  contract. 
Adilitionally,  almost  one-third  did  not  believe 
that  contract  specialists  provided  prompt  re¬ 
sponses  to  their  c|uestions. 


Both  the  contractors  and  the  senior  executives 
were  also  asked  a  series  of  questions  concerning 
the  Federal  procurement  process.  Generally,  the 
views  of  private  contractors  concerning  the  Fed¬ 
eral  procurement  process  were  very  similar  to 
those  held  by  the  SES  members.  As  table  4 
indicates,  both  groups  had  very  little  positive  to 
say  about  the  procurement  process. 

Among  members  of  both  groups  there  was  a 
general  consensus  that  the  Federal  procurement 
process  does  not  effectively  reduce  waste,  fraud, 
or  abuse,  and  does  not  serve  the  best  interest  of 
either  the  Government  or  private  contractors. 
Both  the  contractors  and  the  senior  executives 
were  negative  about  the  procurement  process  in 
several  additional  ways.  For  example,  63  percent 
of  the  contractors  felt  that  the  contracting  pro¬ 
cess  is  too  time  consuming,  and  58  percent  said 
that  the  process  for  awarding  contracts  is  too 
cumbersome.  Similarly,  the  vast  majority  of  SES 
respondents  said  that  the  Federal  procurement 
process  takes  too  long  (91  percent)  and  involves 
too  much  “red  tape”  (89  percent). 

Like  the  contract  specialists  and  supervisors 
discussed  earlier,  both  senior  executives  and 
private  contractors  believed  that  the  main  prob¬ 
lem  is  that  the  Federal  procurement  process  has 
become  far  too  complicated.  From  the  point  of 
view  of  the  senior  executives,  the  complexity  of 
the  Federal  procurement  system  has  simply  ex¬ 
ceeded  the  capabilities  of  the  contract  specialists 
who  support  their  organizations. 

According  to  both  SES  respondents  and  private 
contractors  the  complexity  involved  in  process¬ 
ing  procurement  actions  has  greatly  increased 
over  the  last  10  years.  Unfortunately  these  same 
respondents  did  not  believe  that  the  quality  of  the 
Federal  contract  specialists  has  undergone  a  simi¬ 
lar  increase.  In  fact,  although  contractors  often 
appreciated  attempts  by  contract  specialists  to 
make  things  flow  smoothly,  many  also  thought 
that  most  contract  specialists  were  not  able  or 
adequately  prepared  to  work  effectively,  given 
the  intricacies  of  the  procurement  process. 

From  the  point  of  view  of  members  of  both  client 
groups  this  leaves  the  Government  with  only  two 
alternatives.  According  to  one  respondent,  ei¬ 
ther;  “The  entire  procurement  system  needs  to  be 
simplified  and  made  more  efficient”  or  “tlie  qual¬ 
ity  of  the  people  operating  the  system  has  to 
improve.” 
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TABLE  4 

SES  and  Contractor  Opinions  of  the  Federal  Procurement  Process 

(Percent  of  SES  Members  and  Contractors  Responding  to  the  Question 
"The  Federal  Procurement  Process":) 

SES  Members  Contractors 

Item  Agree  Disagree  Agree  Disagree 

Effectively  reduces  the 
incidence  of  waste,  fraud. 


and  abuse. 

39 

40 

29 

39 

Serves  the  best  interests 

of  the  Government. 

34 

47 

42 

36 

Serves  the  best  interests 
of  private  sector 
contractors. 

25 

45 

11 

65 

Notes:  1 .  The  colums  labeled  "Agree"  include  both  "Strongly  Agree"  and  'Tend  to  Agree" 
responses.  Similarly,  the  "Disagree"  columns  include  both  "Strongly  Disagree"  and  'Tend  to 
Disagree"  responses.  2.  "Neither  Agrev  nor  Disagree"  and  "Don't  lOrow"  responses  were  omitted. 


Conclusions 

Based  upon  the  responses  of  contracting  special¬ 
ists  and  their  supervisors  it  appears  that  the 
members  of  tlie  Federal  procurement  workforce 
are  at  least  basically  qualified  to  perform  their 
jobs.  Contract  specialists  generally  believed  that 
they  possess  the  skills,  knowledges,  and  abilities 
that  they  need,  and  to  some  extent  their  view  is 
substantiated  by  the  large  percentage  of  employ¬ 
ees  who  received  high  peifonnance  appraisal 
ratings.  However,  most  members  of  the  cuirent 
workforce  said  that  additional  training  is  re¬ 
quired,  both  to  perform  critical  tasks  better  and 
to  stay  abreast  of  changes  in  regulations. 

Supervisors  were  also  positive  about  the  capa¬ 
bilities  of  their  subordinates,  but  overall  pre¬ 
sented  a  picture  of  a  somewhat  less  able  workforce 
than  did  the  employees  themselves.  Although 
supervisors  believed  that  their  subordinates  pos- 
.sessed  at  least  acceptable  levels  of  required  skills 
and  abilities,  they  also  thought  that  there  was 
considerable  room  for  improvement.  In  many 
ca.ses,  supervisors  saw  additional  training  as  a 


way  to  improve.  This  was  especially  true  with 
regard  to  the  ability  of  their  subordinates  to 
conduct  negotiations,  to  write,  and  to  be  analyti¬ 
cal. 

Given  the  concerns  of  supervisors,  SES  mem¬ 
bers,  and  private  contractors,  it  is  apparent  that 
tliere  are  significant  problems  and  dissatisfac¬ 
tions  with  the  work  performed  by  contract  spe¬ 
cialists.  The  primary  factor  behind  these  prob¬ 
lems  appears  to  be  the  inability  of  contract  spe¬ 
cialists  to  deal  with  the  increasingly  complicated 
requirements  under  which  they  must  work.  As 
the  number  of  regulations  has  increased,  so  has 
the  complexity  of  job  requirements  and  tlie  amount 
of  time  it  takes  for  a  contract  specialist  to  complete 
many  purchases.  Under  these  conditions  it  is  not 
enough  to  possess  merely  adequate  skills.  Rather, 
to  do  a  good  job  in  today’s  work  environment,  it 
appears  that  most  contract  specialists  need  a  high 
degree  of  both  analytical  ability  and  creativity. 
Unfortunately,  these  were  exactly  the  skills  both 
supervisors  and  members  of  the  SES  believed 
were  most  often  lacking  in  contract  specialists. 
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Recommendations 

How  can  we  correct  this  situation?  Basically, 
MSPB  believes  that  improvement  can  occur  as 
the  result  of  several  actions  which  are  discussed 
in  the  following  four  recommendations. 

Recommendation  1:  Provide  Additional 
Training  to  Improve  the  Quality  of  the 
Current  Workforce 

Based  upon  the  results  of  this  study,  contract 
specialists  need  to  be  better  prepared  to  make  the 
best  possible  business  decisions  for  their  organi¬ 
zations  in  a  timely  manner.  According  to  the 
assessments  provided  by  procurement  supervi¬ 
sors  in  response  to  our  suiwey,  training  should  be 
directed  towards  improving  the  capability  of  the 
workforce  to  conduct  negotiations,  analyze  re¬ 
quirements,  write  clearly,  and  develop  innova¬ 
tive  solutions  to  meet  organizational  needs. 

Training  is  particularly  impoitant  since  almost 
half  of  the  current  procurement  workforce  has 
fewer  than  7  years  of  experience  in  the  occupa¬ 
tional  series.  Moreover,  approximately  three- 
quarters  of  the  people  employed  in  the  GS-1 102 
series  currently  work  for  one  of  the  Defen.se 
agencies.  Since  it  is  clear  that  the  immediate 
future  will  bring  reductions  in  the  size  of  the 
Defense  budget,  it  is  unlikely  that  the  total  Fed¬ 
eral  procurement  workforce  will  continue  to 
grow  as  it  has  since  1981.  For  this  reason,  any 
marked  improvements  in  the  quality  of  the  pro¬ 
curement  workforce  are  not  likely  to  come  from 
large  infusions  of  highly  qualified  new  workers. 
Significant  improvements  in  workforce  quality 
can  only  result  through  an  improvement  in  the 
capabilities  of  current  GS-1 102  incumbents. 

It  is  also  important  that  any  training  in  procure¬ 
ment  that  is  provided  addresses  a  key  issue:  how 
to  make  infonned  business  decisions  that  permit 
an  organization  to  purchase  goods  and  seiwices 
needed  to  meet  mission  objectives  in  a  cost- 
effective  manner.  In  order  to  do  this,  people  need 
to  be  trained  in  the  philosophy  of  procurement. 
This  includes  a  better  understanding  of  the  mis¬ 
sion  of  their  organization  and  the  role  they  play 
in  purchasing  the  goods  and  services  needed  to 
meet  organizational  requirements.  At  some  level 


this  probably  also  involves  an  understanding  of 
the  private  sector  businesses  with  which  they  will 
be  dealing. 

Recommendation  2:  Encourage  and 
Reward  Creativity 

Our  study  found  that  both  employees  and  their 
supervisors  believed  that  most  of  the  people 
working  as  Federal  contract  specialists  have  the 
skills,  knowledges,  and  abilities  needed  to  per¬ 
form  well.  In  the  final  analysis,  however,  the 
customers  for  the  services  provided  by  contract 
specialists  are  often  not  satisfied.  To  some 
extent,  inappropriate  or  inadequate  supervision 
may  contribute  to  this  result. 

Although  supervisors  of  contract  specialists 
criticize  their  subordinates  for  a  lack  of  creativity, 
it  is  the  supervisors  who  must  bear  at  least  some 
of  the  re.sponsibility  for  this  state  of  affairs.  It  is 
the  supervisors  who  set  the  tone  for  the  organi¬ 
zation.  In  many  cases  they  choose  to  reward 
“nile-bound”  approaches  to  contracting  and  dis¬ 
courage  their  subordinates  from  being  more 
creative  and  using  the  flexibilities  that  arecurrently 
available  to  them.  If  contract  specialists  are  to  be 
more  creative,  supervisors  need  to  foster  an 
environment  in  which  employees  are  not  afraid  to 
be  innovative.  Through  their  leadership,  super¬ 
visors  need  to  show  employees  how  they  can  be 
more  responsive  to  their  customers  without  break¬ 
ing  the  rules. 

Recommendation  3:  Ensure  Quality  of  Selec¬ 
tions  Made  for  Entry-Level  Positions 

According  to  many  supeiwisors,  there  has  been  a 
decline  in  the  quality  of  new  entrants  to  the  GS- 
1 102  series  over  the  last  10  to  15  years.  While 
there  is  only  minimal  evidence  to  support  this 
contention,  it  is  clear  that  the  skills,  knowledges, 
and  abilities  needed  by  contract  specialists  have 
increased  in  number  or  complexity.  For  this 
reason,  efforts  designed  to  improve  the  quality  of 
selections  for  GS-1 102  vacancies  are  important. 
Over  time,  an  improvement  in  the  quality  of  the 
people  becoming  contract  specialists  can  have  a 
major  impact  on  the  quality  of  Federal  procure¬ 
ments. 
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In  at  least  the  recent  past,  supervisors  of  contract 
specialists  have  not  always  looked  to  all  possible 
recruitment  sources.  Very  little  use  in  filling 
vacancies  has  been  made  of  the  persons  available 
through  the  Presidential  Management  Intern  Pro¬ 
gram  or  OPM’s  recently  developed  Administra¬ 
tive  Careers  with  America  (ACWA)  procedures. 
In  our  view,  an  improvement  in  the  quality  of  new 
entrants  to  the  GS-1102  series  can  be  made 
through  greater  use  of  the  applicants  who  are 
referred  for  consideration  through  either  use  of 
the  ACWA  procedures  or  the  Presidential  Man¬ 
agement  Intern  Program.  Additionally,  when 
consideringcurrent  Federal  employees  for  place¬ 
ment  into  entry-level  procurement  (GS-1102) 
positions,  managers  should  use  a  wide  area  of 
consideration  m  gather  a  pool  of  high-quality 
applicants  from  a  broad  range  of  occupations  and 
Federal  organizations. 


Recommendation  4:  Where  Possible,  Stream¬ 
line  and  Simplify  the  Procurement  Process 

The  organizations  charged  with  oversight  of  the 
procurement  process  must  constantly  strive  to 
provide  regulatory  and  procedural  safeguards 
against  abuse  while  still  allowing  room  for  the 
exercise  of  reasonable  judgment  on  the  part  of 
contract  specialists.  Contract  specialists,  their 
supervisors,  and  their  clients  all  suggested  that  a 
reasonable  balance  in  this  regard  has  not  been 
achieved  in  that  the  system  tilts  too  heavily 
towards  overly  elaborate  regulatory  require¬ 
ments  and  procedures.  Too  often,  it  was  re¬ 
ported,  procurement  or  contractdecisions  could 
not  be  made  in  the  best  interests  of  the  Govern¬ 
ment  and  the  taxpayer  because  of  the  lack  of 
flexibility.  For  this  reason,  some  simplification 
of  the  procurement  process  could  actually  im¬ 
prove  its  efficiency  and  cost-effectiveness. 
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SUBCONTRACmNG  WITH  EASTERN  EUROPEAN  RESEARCH  FAdLITIES 

-  WHAT  TO  EXPECT 

Fred  Cyran,  P.E.,  Maxima  Technical  Services 


ABSTRACT 

Recent  improvements  in  technology  transfer 
between  the  U.S.  and  Eastern  European 
nations  have  resulted  in  an  "All  Our  Services 
are  For  Sale"  attitude  on  the  part  of  Eastern 
European  enterprises.  As  state-owned 
facilities  have  been  or  are  being  privatized, 
many  directors  of  established  institutions  are 
now  in  the  process  of  learning  how  to 
prepare  proposals  to  win  contracts  in  a 
competitive  environment. 

A  team  venture  was  organized  between  a 
U.S.  firm  and  a  Polish  research  facility  to 
bid  on  a  U.S.  government  contract  involving 
basic  research.  Since  the  Polish  facility 
possessed  a  specialized  expertise  required  by 
the  Request  for  Proposal  (RFP),  the  technical 
aspects  posed  no  problems.  However, 
obtaining  definitive  cost  estimates  and  the 
basis  for  those  estimates  from  the  Polish 
facility  was  somewhat  challenging.  Even 
more  difficult  was  the  necessity  of  tutoring 
the  facility  directors  on  the  philosophy  and 
fundamentals  of  selling  services  to  the  U.S. 
government. 

This  paper  summarizes  the  experience  of  a 
U.S.  design  firm  teaming  together  with  a 
subcontractor  from  an  Eastern  European 
high-tech  research  facility  currently 
undergoing  the  privatization  process.  As  a 
result  of  this  experience,  it  was  concluded 
that  future  relationships  can  be  established 
without  insurmountable  problems  if  adequate 
planning  is  done  prior  to  the  issuance  of  the 
RFP.  In  addition  to  the  need  for  patience,  an 
understanding  of  the  expected  challenges  can 
be  useful  to  U.S.  firms  and  government 


agencies  in  establishing  subcontractor 
support  with  Eastern  Europe. 

INTRODUCTION 

A  small  but  growing  number  of  U.S. 
companies  are  finding  that  Eastern  Europe 
has  strong  research  and  development 
resources  available  at  relatively  low  cost.  A 
well-educated  scientific  community  is  readily 
obtainable.  High-tech  research  facilities  are 
common  in  these  countries.  But  since  only  a 
small  portion  of  the  work  carried  on  in  these 
facilities  during  the  communist  era  ever 
made  it  to  Western  technical  conferences  and 
journals,  many  U.S.  firms  are  unaware  that 
these  facilities  even  exist.  A  large  amount  of 
technical  data  and  publications  from  these 
facilities  also  exists.  This  information  is 
maintained  in  the  libraries  of  these  facilities, 
and  cannot  be  found  through  Western 
literature  searches. 

While  much  of  their  technology  is  behind 
the  state-of-the-art  of  the  West,  there  are 
often  isolated  areas  of  technological  research 
or  capabilities  that  exceed  that  found  in  the 
U.S.  Historically,  research  efforts  in  the 
former  Soviet  bloc  were  not  based  on  a 
profit  motive.  Instead,  they  followed  a  more 
academic  interest  in  their  area  of 
specialization.  They  have  a  keen  interest  in 
fundamental  science  and  understanding 
physical  phenomena.  Often  their  research 
was  aimed  at  solving  practical  problems,  as 
opposed  to  innovating  new  technologies. 

Eastern  European  revsearch  facilities  have  the 
same  attitude  towards  the  West  as  most 
other  enterprises.  They  desperately  need 
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capital  and  want  better  relationship  with  their 
Western  counterparts.  They  are  very 
receptive  to  building  international  exchange 
programs,  joint  research,  and  a  general 
exchange  of  ideas. 

Many  facilities  are  fighting  for  their 
financial  survival.  In  the  past,  some  were 
completely  subsidized  by  the  government. 
As  market  economics  are  being  gradually 
phased  in,  government  subsidies  are  being 
lowered  or  phased  out  entirely.  Directors  of 
these  facilities  are  learning  by  their  own 
experience  the  importance  of  marketing  their 
services  to  secure  U.S.  funded  work.  These 
institutions  are  not  looking  for  handouts. 
Rather,  they  want  to  jointly  develop  research 
opportunities,  and  to  acquire  state-of-the-art 
research  equipment.  This  creates  favorable 
fee  ranges.  Sometimes  attractive  barter 
arrangement  are  available  (usually  their 
brainpower  for  Western  computers  and  high- 
tech  hardware).  It  is  not  unusual  to  procure 
quality  contract  research  at  half  the  co.st  of 
doing  similar  work  in  the  U.S. 

Quality  work  at  low  cost  is  not  without  it's 
problems  in  Eastern  Europe.  Many  of  the 
most  basic  principles  of  project  management, 
instinctive  to  U.S.  firms,  are  seriously 
lacking  at  all  levels.  These  shortcomings  can 
occur  in  the  form  of  cost  estimates  without 
basis,  inability  to  anticipate  factors  affecting 
schedule,  and  a  tendency  to  "drift"  off  of 
technical  objectives  in  pursuit  of  tangent 
interests.  But  what  they  lack  in  management, 
they  make  up  for  in  superior  technical 
capabilities.  This  is  the  key  to  a  successful 
joint  research  project  -  U.S.  management 
leadership,  coupled  with  their  excellent 
scientific  aptitude. 

ITie  opportunities  of  Ikastern  European 
facilities  is  by  no  means  a  buyer's  paradi.se. 
Many  joint  research  projects  fail  because  of 
communication  gaps  and  a  lack  of 


understanding  about  what  can  be  realistically 
accomplished.  A  successful  project  in 
Eastern  Europe  requires  a  good 
understanding  of  the  culture,  attention  to 
detail,  adequate  planning,  and  the 
determination  to  establish  a  long  term 
relationship.  This  is  not  an  environment  to 
expect  quick  profits  or  breakthroughs.  But 
finding  a  common  research  interest  with 
them  can  reap  handsome  dividends. 

The  best  method  of  informing  U.S.  firms  of 
what  to  expect  from  working  with  Eastern 
Europe  facilities  is  by  citing  a  recent 
experience  on  a  joint  project.  'The 
information  contained  in  this  paper  is  based 
largely  on  a  U.S.  aerospace  firm's 
subcontracting  relationship  with  the  Polish 
Aviation  Institute  (Instytut  Lotnictwa)  in 
Warsaw,  a  national  aeronautical  research 
facility.  The  technical  subject  involved 
contract  research  in  the  are  •  of  i  .iiovative 
low-speed  wing.s  surfa.r  'utours, 
fabrication  of  wind  tunnel  inoucis,  and 
support  of  testing  their  theories  in  U.S. 
facilities,  Tlie  work  was  being  procured  by 
a  U.S.  government  agency  via  a  Request  for 
Proposal  (REP). 

ESTABLISHING  CONTACT 

Finding  a  suitable  research  facility  in  Eastern 
Europe  is  generally  not  very  difficult. 
However,  the  topic  of  locating  such  a  facility 
is  beyond  the  scope  of  this  paper.  Numerous 
consultants  arc  available  in  the  U.S.  who 
specialize  in  technology  transfer  with 
Eastern  Europe  and  in  locating  industries  or 
state-related  agencies  that  conduct  high-tech 
research.'  Once  a  candidate  facility  is 
located,  establishing  a  dialogue  is  relatively 
easy.  The  next  step  is  to  write  or  fax  a  letter 
of  inquiry  to  them  directly,  in  the  English 
language,  addressed  to  the  facility  director. 
They  will  typically  respond  with  an  English 
response  letter  giving  the  name  of  an 
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English  speaking  contact  at  their  facility  and 
a  short,  general  flyer  on  their  capabilities.  As 
their  postal  system  and  internal  bureaucratic 
organization  is  generally  inefficient,  it  is  not 
uncommon  for  letters  or  faxes  to  be  lost  in 
transit.  Generally  sending  information  by  fax 
is  more  effective  than  mailing  letters.  Do  not 
be  discouraged  if  two  or  three 
correspondences  are  needed  before  you  get 
an  answer.  However,  once  an  appropriate 
contact  in  their  organization  is  identified, 
communication  will  usually  improve.  Initial 
contact  will  be  faster  and  more  effective  if  a 
consulting  firm  is  used,  or  if  the  U.S. 
organization  has  personal  knowledge  of  an 
individual  working  in  their  field  of  research. 

NECESSITY  OF  SITE  VISITS 

In  the  case  of  the  author's  relationship  with 
the  Polish  Aviation  Institute,  previous 
contact  and  visits  were  established  prior  to 
the  release  of  an  RFP.  Had  a  detailed 
description  of  their  capabilities  not  been 
known,  establishing  a  relationship  in  the 
short  period  needed  to  respond  to  an  RFP 
would  not  have  been  possible. 

Typical  Eastern  European  research  facilities 
have  relatively  poor  and  unorganized 
marketing  literature  on  their  capabilities.  All 
facilities  have  some  infonnation,  often  in 
English.  However,  they  do  not  have  much 
experience  in  providing  colorful  and 
appealing  sales  literature.  Tlieir  translation 
into  English  may  be  less  than  desirable. 
Often  their  statements  of  capabilities  are 
prepared  on  a  typewriter  and  duplicated  in 
ma.ss.  Such  literature  is  still  valuable  for  the 
technical  specialist,  who  can  sift  through  the 
information  to  determine  if  the  facility  has 
potential  capabilities. 

A  visit  to  their  facility  is  a  necessity. 
Language  should  never  be  considered  a 
problem.  Virtually  all  facilities  have  at  least 


one  person  on  their  staff  who  speaks  English 
and  is  knowledgeable  of  their  area  of 
specialty.  This  is  normally  sufficient  to 
conduct  a  tour  and  discussion  of  past 
projects  and  capabilities.  If  the  U.S.  firm 
knows  the  subject  matter  well,  technical 
communication  should  not  be  a  problem. 
Cordial,  hospitable  visits  are  the  norm, 
becau.se  of  eagerness  to  e.stablish  ties  with 
the  U.S.  They  are  usually  generous  in 
providing  samples  of  their  work,  along  with 
an  extensive  tour  of  their  facilities.  They  are 
also  eager  to  introduce  their  staff  and  present 
their  latest  research.  In  fact,  it  would  not  be 
unusual  if  they  reveal  more  about  their 
strengths  and  weakness  than  would  be 
expected  from  an  American  firm. 

A  visit  to  an  Eastern  European  facility, 
especially  those  previously  state-owned,  can 
be  discouraging.  Much  of  their  equipment  is 
vintage  1940-50  in  both  style  and  function, 
but  is  usually  functional  and  well 
maintained.  Scattered  throughout  the  facility 
will  be  selected  high-tech  equipment 
produced  in  the  last  20  years.  Data 
acquisition  equipment,  for  example, 
generally  does  not  compare  with  the  West. 
However,  it  is  usually  simple  and  practical. 
It  will  be  obvious  that  little  money  was 
available  to  procure  the  latest  equipment. 
The  humble  appearance  of  the  site  must  be 
overlooked  to  achieve  your  objectives: 
finding  a  specific  technology  where  they 
have  expertise  in  spite  of  their  equipment 
shortcomings.  Whereas  the  overall  facility 
may  be  obsolete  by  Western  standards,  there 
may  be  a  few  areas  which  are  highly 
advanced. 

TEAMING 

Once  an  area  of  rt  .^nrch  has  been  identified, 
the  {X)tentiai  applications  should  be 
di.scusscd  with  the  technical  management  of 
the  facility  to  determine  their  interest,  and 
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more  importantly,  to  begin  the  formation  of 
a  research  team.  In  some  cases  they  may 
have  had  joint  projects  with  other  facilities, 
either  in  their  own  country  or  other  facilities 
in  the  Soviet  bloc.  They  may  be  hesitant  to 
mention  other  contact  or  inquiries  with 
Western  firms  for  fear  of  damaging  their 
relationship  with  them.  Inquiries  and 
involvement  with  Western  firms  are 
probably  more  extensive  than  commonly 
believed.  But  since  many  inquiries  do  not 
result  in  establishing  a  joint  project,  they 
may  not  want  to  openly  discuss  them. 

It  is  also  typical  that  your  initial  relationship 
with  them  may  cool  somewhat  after  your 
site  visit.  This  stems  from  their  expectations 
that  you  want  to  engage  them  immediately 
for  a  specific  work  effort.  If  they  perceive 
that  you  are  only  considering  potential  work, 
as  opposed  to  an  aggressive  interest  to  retain 
them,  them  may  lose  interest  and  be 
unresponsive.  This  attitude  results  from  their 
more  common  dealing  with  Western 
European  firms  (typically  German,  Dutch, 
Austrian  and  French),  who  deal  with  them 
differently  than  their  U.S.  counterparts.  They 
may  also  be  unresponsive  if  bureaucratic 
delay  occur  in  your  planning  or  funding 
authorization. 

When  suitable  capabilities  are  identified,  and 
candidate  researchers  have  been  identified  by 
name,  a  short  (1-2  page)  letter  of  intent 
should  be  submitted  from  your  organization 
to  their  facility.  They  should  respond  back 
with  a  letter  from  the  facility  director  that 
they  intend  to  support  the  program  you 
describe,  with  the  people  previously  named, 
and  that  they  are  willing  to  enter  into 
contractual  negotiations  to  identify  the 
detailed  scope,  .schedule  and  budget  within  a 
fixed  period  of  time. 

If  the  candidate  project  involves  the 
submission  of  a  proposal  in  response  to  a 


U.S.  government  RFP,  a  significant  effort 
will  be  needed  to  explain  the  U.S. 
government  procurement  process  to  them. 
Unless  their  portion  of  the  work  is  small, 
there  will  probably  not  be  enough  time  to 
educate  them  on  the  basics  of  the  process.  It 
is  difficult  for  them  to  comprehend  the 
competitive  nature  of  proposal  submission.  It 
is  even  more  difficult  to  get  their  full  and 
timely  support  on  a  proposal  they  may  not 
win.  If  they  do  give  their  full  support,  they 
may  assume  that  a  contract  is  assured  - 
losing  sight  of  the  possibility  that  award  may 
be  delayed,  canceled  or  lost  to  another 
bidder.  Such  an  experience  will  discourage 
further  association  with  your  organization. 
To  alleviate  this  kind  of  a  situation,  the  need 
to  educate  the  management  of  the  Eastern 
European  facility  on  standard  U.S. 
government  protocols  can  not  be 
overemphasized. 

Throughout  your  entire  correspondence  and 
teaming  efforts,  an  aggressive,  pro-active 
attitude  must  be  maintained  towards  them. 
They  are  typically  reactive,  and  function 
well  if  presented  with  a  specific  technical 
problem  needing  solution,  rather  than  asking 
them  for  conceptional  studies,  proposals  or 
administrative  data.  They  will  also  be  very 
disinterested  in  your  project  if  you  expect 
them  to  respond  with  elaborate  propo.sals 
describing  non-technical  issue  (as  they 
perceive  them).  You  will  get  a  better 
response  if  you  ask  for  a  specific  solution  to 
a  technical  problem.  If  you  .say  "This  is  the 
problem  we  have.  Can  you  solve  our 
problem  or  determine  what  is  causing  it", 
you  will  get  a  better  response  than  if  you 
say  "Could  you  propose  several  concepts  in 
this  particular  area  where  you  are  interested 
in  working,  and  provide  a  preliminary  plan 
on  how  to  accomplish  this  work"?  In  all 
your  communications,  it  must  be 
remembered  that  their  great  strength  is  in 
technical  problem  solving.  To  be  succe.ssful 
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the  U.S.  firm  must  take  the  lead  in 
managerial  and  administrative  matters. 

COMMITMENT 

A  common  discouragement  to  U.S.  firms  is 
talk  without  action.  Additional  written 
information,  future  appointments,  output 
samples  and  follow-ups  to  action  items  are 
sometimes  promised  and  not  delivered.  This 
is  not  intentional,  but  is  rooted  in  living 
under  the  former  highly  bureaucratic 
communist  system.  They  may  repeatedly 
mention  their  desire  to  do  business  with  you, 
but  won't  seem  to  take  any  action  on  their 
part  to  initiate  such  work.  When  they  say 
they  desire  to  do  business  with  you,  they 
really  mean  it.  But  hidden  in  their 
enthusiasm,  there  will  often  be  a  plea  for 
you  to  tell  them  specifically  what  needs  to 
be  done  to  make  this  a  reality.  Another 
frustration  is  the  "it's  in  the  mail"  response. 
Often  this  is  true.  They  may  tell  their 
secretary  to  mail  certain  material  to  you, 
which  is  subsequently  mailed  via  the 
slowest,  least  costly  means.  Also,  directors 
and  administrators  may  change  suddenly, 
with  no  transfer  of  knowledge  of  current 
activities  between  them  and  their  successors, 
causing  additional  frustration. 

To  be  effective,  it  may  help  to  consider 
some  aspects  of  what  business  was  like  in 
the  U.S.  in  the  1930's.  For  e.xample,  they 
react  favorably  to  the  old  fashion  method  of 
dropping  in  on  them,  in  person,  to  conduct 
business.  This  is  far  more  effective  that 
doing  business  via  mail  or  telephone. 
Perhaps  you  will  find  the  right  people  there 
at  that  time.  Perhaps  not.  But  you  will 
probably  have  the  same  success  as  if  you 
scheduled  appointments  with  them  in 
advance.  Team  members  from  the  U.S.  may 
have  to  adjust  to  this  style  of  conducting 
business,  which  is  generally  the  opposite  of 
what  Americans  e.xpect  or  are  accustomed 


to.  Retaining  the  services  of  an  .American 
representative  in  the  Eastern  European 
country  is  also  recommended.  This  will  help 
you  understand  the  business  and  cultural 
practices,  minimize  frustration,  save  time 
and  give  you  strategic  advice  leading 
towards  constructive  negotiations  with  their 
facility. 

Commitments  for  specific  projects  must  be 
negotiated  in  person,  preferably  at  their  site. 
A  general  rule  to  follow  in  estimating  how 
much  time  it  will  take  to  scope  the  work  and 
prepare  a  plan  for  joint  cooperation  is  2-.3 
times  the  amount  of  time  required  to  do 
.similar  preparations  in  the  U.S.  Normally 
this  w'ill  invol\;e  multiple  visits  of  multiple 
days  at  a  time.^"  You  mu.st  demonstrate  that 
you  are  willing  to  take  the  lead,  and  need 
their  support  to  make  the  project  successful. 

You  will  need  to  prepare  the  bulk  of  their 
propo.sal.  This  is  best  done  by  submitting  to 
them  a  draft  for  the  technical  scope  of  work 
you  expect  them  to  do,  along  with  a  work 
breakdown  structure  on  how  you  envision 
them  to  accomplish  the  work. 

Some  principles  to  observe  during  the 
commitment  stage: 

o  It  is  essential  to  keep  the  definition 

of  what  research  is  expected  as 
simple  as  possible.  A  excessively 
complicated  project  has  a  much 
greater  chance  of  failure. 

o  Consider  only  technologies  in  which 

both  your  organization  and  the 
Eastern  European  facility  have 
expertise. 

o  Do  not  be  discouraged  by  their  lack 

of  busine.ss  tact.  It  is  unrealistic  to 
expect  that  their  preparation, 
organization,  and  courtesy  will  be 
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similar  to  a  U.S.  firm.  They  may  not 
be  bashful  in  e.xpressing  their  own 
wants  and  desires,  without  any 
empathy  for  your  organization's 
objectives. 

COST  ESTIMATES 

You  should  not  expect  to  obtain  reasonable 
cost  estimates  unless  you  take  an  active  role 
in  preparing  the  estimates  with  them.  In 
most  cases,  there  will  be  no  basis  for  the 
estimated  costs.  Historical  data  and  records 
of  prior  cost  data  simply  do  not  exist,  or  will 
not  be  representative  of  their  true  cost  from 
a  Western  perspective.  Fortunately,  many 
U.S.  accounting  firms  have  offices  in 
Eastern  Europe,  supporting  the  large  demand 
to  assist  local  businesses  in  a.ssembling  or 
reconstructing  needed  information  for  cash 
flow  and  balance  sheet  data.  These 
accounting  firms  can  be  of  great  assistance 
during  the  estimating  proce.ss. 

The  author's  first  experience  with  the  cost 
estimating  process  of  the  Polish  Aviation 
Institute  was  interesting.  When  the  subject  of 
estimating  the  cost  of  their  services  arose,  an 
uncomfortable  feeling  came  over  the 
meeting,  as  if  a  sensitive  issue  was  raised 
concerning  their  integrity.  To  diffuse  the 
situation,  sample  cost  estimate  spreadsheets 
were  presented.  The  proce.ss  of  estimating 
the  resources  tor  individual  work  elements 
was  explained,  by  showing  manhours  with 
accompanying  labor  rates,  material  co.sts, 
overhead,  and  resource  of  totals.  Their 
management  was  not  familiar  with  this  kind 
of  a  detailed  cost  estimate  format.  After 
about  10  minutes  of  explaining  the  basics  of 
developing  such  an  estimate,  the  mood  of 
the  meeting  became  pleasant.  That  portion  of 
the  meeting  ended  with  the  response  "Sure, 
we  can  do  that.  We'll  just  follow  your 
format. "  f  hese  decision  makers  were  people 
with  highly  competent  technical  backgrounds 


in  their  area  of  specialization.  They  knew 
their  technical  subjects.  But  they  needed  and 
expected  managerial  leadership  from  the 
American  partners. 

To  start  the  estimating  process,  a  general 
idea  of  what  it  will  cost  to  do  the  similar 
work  in  the  U.S.  is  essential.  Without  this 
reference  point,  many  directors  of  technical 
facilities  will  quote  you  a  price  based  on 
what  they  think  you  would  be  willing  to  pay 
them,  and  not  on  any  basis  of  their  internal 
cost.  The  following  guidelines  should  be 
useful  to  assist  in  the  development  of  a  cost 
estimate: 

o  Procure  a  general  estimate  of  what  it 
would  cost  to  perform  the  entire  job 
in  the  U.S. 

o  Cut  the  U.S.  estimate  in  half 

o  Propose  that  you  have  this  amount  of 
funds  available  to  allocate  to  their 
facility,  and  ask  what  they  can  give 
you  towards  meeting  your 
requirements. 

o  When  they  agree  to  do  the  work  for 
the  amount  you  propose,  you  must 
initiate  a  work  breakdown  structure 
with  them.  This  must  include  both 
manhours  for  the  personnel  they  plan 
to  assign  to  the  project,  and 
identification  of  the  kinds  of 
materials.  Let  them  assign  manhour 
rates  and  cost  for  the  materials. 

o  Don't  expect  to  receive  any 
quantitative  basis  for  labor  costs. 
Depending  on  their  receptiveness, 
they  will  alter  the  cost  to  the  extent 
they  want  the  work. 

o  When  you  have  prepared  a  rough- 
draft  spreadsheet,  identifying  the 
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work  breakdown  structure  and 
calculated  cost,  total  the  complete 
estimate,  review  the  intended  vScope 
of  work,  and  negotiate  a  preliminary 
commitment  that  they  can  do  the 
work  on  schedule  and  at  the 
estimated  cost. 

o  You  will  need  to  do  essentially  all 
the  preparation  of  documents  on  the 
scope  of  work  and  the  cost  estimate. 
The  amount  of  funds  you  intend  to 
pay  for  their  work  and  method  of 
payment  will  also  need  to  be  clearly 
documented.  You  may  be 
disappointed  if  you  wait  for  them  to 
respond  with  a  written  response  to 
your  request.  They  are  iot  typically 
proposal  writers;  they  will  look  to 
you  to  take  the  lead  in  the  initiation 
and  generation  of  contract  and 
planning  documents. 

o  Identification  of  specific  deliverable 
milestones  is  critical.  It  must  be 
made  clear  that  specific  penalties  will 
be  a-ssessed  if  these  milestones  are 
not  met.  Unless  you  have  a  good 
prior  relation.ship  with  them, 
incremental  payments  for  their  work 
normally  should  be  tied  to  these 
milestones. 

DECISION  MAKERS 

Facility  directors  at  most  Ea.stern  European 
research  facilities  have  significantly  more 
authority  to  enter  into  contractual 
arrangements  and  to  set  prices  than  their 
American  counterparts.  At  similar  U.S. 
facilities,  authorization  is  generally  needed 
from  a  level  higher  than  the  facility  director. 
Such  approval  cycles  may  cause  significant 
delays  in  e.xecution  of  a  contact,  which  may 
cause  them  to  question  the  commitment  of 
the  U.S.  partner.  The  same  applies  to  price 


setting.  If  the  facility  director  is  eager  for 
the  work,  he  can  negotiate  the  deal  on  his 
own  authority.  U.S.  partners  are  sometimes 
surprised  at  the  simplicity  of  their  sales 
process.  They  would  be  ready  and  eager  to 
accommodate  you  if  you  arrived  with  a 
suitcase  of  American  currency  and  wanted  to 
buy  hardware  or  services  on  the  spot. 

Depending  on  the  personal  contacts  of  the 
facility  director  with  key  personnel  on 
government  legislative  committees,  leverage 
of  matching  funding  may  be  available  if  the 
scope  of  the  re.search  falls  within  national 
priority  guidelines.  In  Poland,  for  example, 
technology  that  will  improve  the  export 
marketability  of  food  products  could  be 
given  subsidies  from  the  government.  This 
may  be  in  the  form  of  funding  to  purchase 
equipment  or  building  new  facilities  capable 
of  meeting  the  U.S.  firm's  requirements. 

On  the  U.S.  side,  leverage  may  be  available 
in  the  form  of  providing  their  facility  with 
equipment  that  is  slightly  older  than  state-of- 
the-art.  This  could  be  good  hardware  that  the 
U.S.  firms  plans  to  replace  with  newer 
equipment.  However,  credibility  could  be 
lost  if  you  offer  outdated  or  low  quality 
hardware.  Although  they  may  not  have  it  in 
their  facility,  they  know  and  covet  quality 
equipment. 

FINALIZING  A  CONTRACT 

The  final  proce.ss  of  entering  into  a  contract 
with  an  Eastern  European  facility  will 
involve  the  signing  of  a  detailed  contract,  the 
discussion  of  which  is  beyond  the  scope  of 
this  paper.  It  is  assumed  that  throughout  the 
acquisition  process,  services  will  be  needed 
from  lawyers,  bankers,  accountants,  contract 
administrators,  translators,  and  export 
licensing  experts,  who  are  familiar  with 
dealing  with  Eastern  Europe.  With  the 
exception  of  banking,  none  of  these  support 
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services  should  present  formidable  problems, 
compared  with  the  technical  issues  of 
finding  and  identifying  beneficial  joint 
projects.  Banking  and  currency  transfers  can 
be  major  problems,  depending  on  the 
particular  terms  of  the  contract.  This  needs 
to  be  considered  in  the  early  part  of  the 
negotiation  process. 

CONCLUSION/SUMMARY 

Eastern  Europe  offers  a  highly  skilled  labor 
force  and  high-tech  research  facilities  at 
relatively  low  cost.  U.S  firms  can  get  an 
excellent  return  on  their  investment  by 
taking  advantage  of  this  abundant  problem¬ 
solving  brainpower.  Much  of  their 
capabilities  are  not  well  known  in  the  U.S., 
stemming  in  part  from  lack  of  experience  in 
marketing  their  services  to  the  West,  and 
from  an  inefficient  infrastructure  set  in  place 
by  the  former  communist  government. 

In  implementing  economic  reforms  and 
establishing  a  market  economy,  virtually  all 
industries  and  formerly  state-supported 
research  facilities  are  offering  their  services 
to  the  West  to  earn  hard  currency.  If  a 
suitable  area  of  interest  can  be  identified 
between  an  American  firm  and  an  Eastern 
European  research  facility,  infrastructure  and 
lack  of  experience  should  not  discourage  the 
formation  of  a  profitable  relationship.  The 
key  to  a  successful  joint  research  project  is 
to  combine  U.S.  management  capabilities 
with  Eastern  European  technical  expertise. 
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ABSTRACT 

The  Manpower,  Personnel,  and  Training 
(MPT)  in  Acquisition  Decision  Support 
System  (DSS)  is  an  Air  Force  program 
providing  the  first  integrated  tool  for 
addressing  MPT  requirements  during  system 
acquisition  and  design.  New  weapon  system 
development  and  major  modifications  have 
historically  neglected  how  our  most  impor¬ 
tant  and  costly  resource  -  people  -  will 
maintain  and  support  the  fielded  system. 
Inadequate  planning  for  training  and  deploy¬ 
ing  the  human  element  has  often  delayed 
system  operational  dates.  This  DSS  will 
assist  acquisition  managers  and  analysts  to 
effectively  integrate  people  issues  (numbers, 
characteristics,  proficiency)  with  equipment 
(aircraft)  early  in  the  acquisition  cycle. 
Acquisition  specialists  can  use  the  structured 
analysis  approach  provided  by  the  MPl 
DSS  to  ensure  that  system  people  costs  are 
affordable,  jobs  are  properly  structured,  and 
people  are  trained  prior  to  the  system 
becoming  operational.  The  MPT  DSS  is 
being  designed  to  support  the  Human 
System  Integration  requirements,  now 
directed  under  DOD  Instruction  5000.2. 

INTRODUCTION 

New  defense  systems  acquisition  or  major 
modifications  have  historically  focused  on 
system  costs,  schedule,  performance,  and  in 
recent  years  -  logistics  support.  Unfortu¬ 


nately,  the  human  element  has  always  been 
left  for  last.  Human  factors  experts  have 
made  strides  to  enhance  performance  and 
logistics  maintenance  work,  but  how  we 
employ  our  people  (recruiting,  job  descrip¬ 
tions,  personnel  abilities,  training,  organiza¬ 
tional  responsibilities)  and  their  associated 
costs  are  often  neglected.  The  opportunities 
to  optimize  the  human  centered  elements  are 
enormous.  By  considering  the  hiunan 
capabilities  and  limitations,  beginning  with 
weapon  system  conceptualization,  the 
human  can  be  eliminated  as  the  factor  which 
currently  restricts  combat  capability; 
systems  can  be  maintained  faster,  smarter, 
and  cheaper;  people  can  be  trained  better,  in 
less  time,  with  higher  efficiency;  systems 
can  be  made  safer  for  the  operator,  the 
maintainer,  and  the  non-combat  environ¬ 
ment.  All  of  this  can  be  achieved  by  influ¬ 
encing  the  design  of  the  weapon  system  to 
enhance  the  combat  capability  through 
economies  of  our  human  resources  [1]. 

In  this  era  of  decreasing  defense  budgets, 
each  system  is  coming  imder  increasing 
scrutiny  concerning  mission  need,  system 
requirements,  logistics  support,  and  life- 
cycle  costs  (LCCs)  by  both  Congress  and 
the  Department  of  Defense  (DoD)  [2].  This 
scrutiny  drives  our  need  to  economize  the 
way  we  employ  our  human  resources  to 
achieve  the  best  people-to-system  tradeoff 
we  can  obtain.  Every  system  requires  people 
to  operate,  maintain,  and  support  it.  An  Air 
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Force  cost  study  showed  that  up  to  60 
percent  of  an  aircraft's  yearly  operations  and 
support  costs  can  be  directly  attributed  to 
manpower,  personnel,  and  training  cost 
elements  [3].  As  the  Air  Force's  manpower 
authorizations  continue  to  shrink,  and  per¬ 
sonnel  compensation  increases,  this  trend  is 
likely  to  increase  unless 
new  system  designs  are 
influenced  and  adjusted 
to  make  the  systems 
easier  to  operate,  main¬ 
tain,  and  support  with 
fewer  people  at  existing 
skill  levels.  Early  iden¬ 
tification  of  manpower, 
personnel,  training,  and 
safety  (MPTS)  high 
drivers,  goals,  con¬ 
straints,  and  issues  can 
provide  positive  design 
influences  for  new  wea¬ 
pon  systems  if  properly 
integrated  into  the  acqui¬ 
sition  and  engineering 
process. 


Congress  and  the  Department  of  Defense 
recognized  the  need  for  better  integration  of 
the  human  during  weapon  system's  acquisi¬ 
tions.  Department  of  Defense  Directive 
(DODD)  5000.1  and  it's  associated  Instruc¬ 
tion  DODI  5000.2  have  defined  a  Human 
Systems  Integration  Program  [4].  The 
policies  and  procedures  established  in  this 
instruction  serve  as  the  basis  for  integrating 
human  factors  engineering,  manpower, 
personnel,  training,  health  hazards,  and 
safety  considerations  into  the  acquisition  of 
defense  systems.  The  policies  set  forth  are 
"1)  human  considerations  shall  be 
effectively  integrated  into  the  design  effort 
for  defense  systems  to  improve  total  system 
performance  and  reduce  costs  of  ownership 
by  focusing  attention  on  the  capabilities  and 
limitations  of  the  soldier,  sailor,  airman,  or 
marine.  2)  Objectives  for  the  human 
element  of  the  system  shall  be  initially 
established  at  Milestone  I,  Concept 


Demonstration  Approval,  and  be  traceable 
to  readiness,  force  structure,  affordability, 
and  wartime  operational  objectives.  They 
shall  be  subsequently  refined  and  updated  at 
successive  milestone  decision  points." 

The  U.S.  Air  Force's  implementation  of 


these  procedures  is  embodied  in  the 
Integrated  Manpower,  Personnel,  and  Com¬ 
prehensive  Training,  and  Safety 
(IMPACTS)  Program  (AFR  26-1).  Just  as 
the  Army's  Manpower  and  Personnel  Inte¬ 
gration  (MANPRTNT)  program  and  the 
Navy's  Hardware  versus  Manpower 
(HARDMAN)  integration  program  look  at 
integrating  soldiers  and  sailors  into  defense 
systems  that  are  peculiar  to  those  services, 
the  IMPACTS  program  emphasizes  integrat¬ 
ing  airman  into  air,  space,  and  ground 
support  defense  systems  within  the  Air 
Force  organization.  The  IMPACTS  program 
consists  of  a  policy  arm,  trainers,  practitio¬ 
ners,  and  continuing  research  (Figure  1)  to 
improve  IMPACTS  processes  and  methods. 
This  paper  addresses  the  research  arm  of  the 
IMPACTS  program  and  describes  an  inte¬ 
grated  analysis  system  now  being  devel¬ 
oped.  IMPACTS  analysts  and  acquisition 
managers  will  use  this  analysis  system  to 
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ensure  weapon  system  MPT  costs  are 
affordable,  jobs  are  properly  structured,  and 
people  are  trained  for  their  jobs  prior  to  a 
new  system  becoming  operational. 

MPT  Research  Program 

Manpower,  personnel,  and  training  (MPT) 
analysis  evaluates  human-in-loop  costs  and 
capabilities  with  intent  to  minimize  MPT 
related  LCCs  while  maximizing  system 
capabilities.  To  clarify  terminology,  man¬ 
power,  refers  to  the  number  of  positions 
needed  to  operate,  maintain,  and  support  a 
system  in  its  operational  environment; 
personnel,  to  the  types  of  people  required 
and  their  characteristics  and  skills;  and 
training,  to  what  they  need  to  know  to  do 
their  job  and  what  resources  (trainers  and 
training  systems)  will  be  required  to  achieve 
the  desired  skill  proficiency.  "This  amoimts 
to  sizing  (M),  describing  (P),  and  enabling 
(T)  the  work  force  so  that  it  can  accomplish 
a  given  workload  or  function  effectively  and 
economically"  [5]. 

To  provide  the  best  new,  or  modified, 
weapon  system  at  the  least  LCC,  decision 
makers  need  up-to-date  data  and  analysis 
tools.  The  first  Air  Force-wide  MPT  con¬ 
ference  [6]  held  in  May  1987  identified  that 
a  major  problem  within  the  acquisition  com¬ 
munity  was  (and  continues  to  be)  the  lack  of 
an  integrated  database  and  analysis  meth¬ 
odologies  to  effectively  analyze  interrelated 
MPT  issues.  The  Air  Force  Human 
Resources  Laboratory  (AFHRL  —  now 
Armstrong  Laboratory,  Human  Resources 
Directorate)  laimched  a  comprehensive 
research  program  to  meet  the  needs  of 
System  Program  Office  (SPO)  decision 
makers  in  the  acquisition  process  [7].  This 
program  was  designed  to  investigate  data 
and  data  sources  that  could  be  used  to 
support  MPT  analyses  and  began  developing 
methods  and  tools  in  each  of  the  functional 
domains  to  exploit  the  data.  The  objective 
was  to  develop  a  collection  of  methods  and 
prototype  tools  that  could  eventually  be 


integrated  into  a  single  integrated  decision 
support  system.  The  integrated  system 
would  enable  acquisition  and  operational 
analysts  to  demonstrate  the  MPT  related 
costs  associated  with  various  proposed 
weapon  system  designs,  thus  allowing 
design  tradeoffs  to  reduce  life-cycle  costs. 

The  research  culminated  in  the  MPT  Inte¬ 
gration  Branch  awarding  a  four  year, 
multimillion  dollar  advanced  research  and 
development  (R&D)  contract  to  provide 
acquisition  decision  makers  with  just  such  a 
DSS  [8].  The  contract  to  develop  a 
Prototype  Manpower,  Personnel,  and 
Training  (MPT)  in  Acquisition  Decision 
Support  System  (DSS)  was  awarded  (Feb 
92)  to  Dynamics  Research  Corporation  and 
their  subcontractors;  Micro  Analysis  and 
Design,  Rishi  Technologies,  and 
Organizational  Research  and  Development. 
These  companies  have  researchers  who  are 
intimately  familiar  with  both  the  acquisition 
process  and  MPT  issues  and  possess  the 
needed  knowledge  and  skilled  staff 
capability  to  successfully  develop  an 
integrated  system. 

MPT  in  Acquisition  DSS 

This  Advanced  Technology  Transition 
Development  program  will  provide  the  first 
integrated  tool  for  addressing  MPT  require¬ 
ments  during  system  acquisition  and  design. 
The  MPT  DSS  is  a  micro-level  (job  task 
level)  tool  that  will  help  analysts  build  a 
credible  baseline  of  measurable  MPT  goals 
and  constraints,  provide  MPT  inputs  needed 
for  system  tradeoff  studies,  allow  analysts  to 
study  design  alternative  implications,  and 
verify  whether  the  completed  system  devel¬ 
opment  achieved  the  MPT  goals  and 
constraints.  The  system  will  automate  the 
extraction  of  historical  MPT  data  from  Air 
Force  data  bases  and  new  system  data  from 
the  Logistics  Support  Analysis  Record 
(LSAR).  This  historical  data  will  be  used  to 
create  a  baseline  comparison  system.  As 
new  system  information  is  received,  a 


notional  or  proposed  system  configuration 
will  emerge.  Finally,  a  suite  of  MPT 
analysis  methodologies  and  tradeoff  tools 
applied  to  a  baseline  comparison  system 
(BCS)  and  proposed  system  will  produce 
key  MPT  products  needed  to  support  the 
acquisition  and  design  process.  The  purpose 
of  the  MPT  DSS  is  to  reduce  defense 
system's  life-cycle  costs  while  improving 
combat  readiness  and  supportability  by 
identifying  and  resolving  MPT  issues  early 
in  the  acquisition  of  these  systems. 

The  MPT  DSS  is  graphically  depicted  in 
Figure  2.  This  illustrates  the  micro-level 
data  bases  and  types  of  analysis  techniques 
needed  to  provide  a  comprehensive,  inte¬ 
grated  tool.  The  MPT  DSS  will  support  all 
phases  of  the  acquisition  process,  from 
requirements  analysis  and  determination  at 
the  Air  Force  major  commands 
(MAJCOMs)  to  design  evaluation  in  the 


SPOs,  Primary  analysis  goals  are  to  validate 
that  emerging  weapon  system  designs  meet 
MPT  constraints  imposed  on  that  system 
and  to  provide  personnel  and  training  plan¬ 
ners  with  information  and  decision  proc¬ 
esses  to  establish  efficient  training  and 
personnel  pipelines  before  weapon  system 
delivery. 

The  MPT  DSS  is  based  on  the  results  of  a 
Manpower,  Personnel,  Training,  and  Safety 
(MPTS)  factors  in  the  system  acquisition 
process  study  completed  for  the  Human 
Systems  Division  (HSD)  [9],  a  front  end 
analysis  of  an  MPT  modeling  architecture 
[10],  and  an  evaluation  of  the  Army's 
Hardware  versus  Manpower  methodology 
(HARDMAN  III)  suite  of  MPT  tools  [11]. 
Continuing  close  coordination  with  Army 
HARDMAN  experts  is  ensuring  compatibil¬ 
ity  between  the  tools  and  avoiding  duplica¬ 
tion  of  effort. 
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The  unique  capability  distinguishing  the 
MPT  DSS  from  Army  and  Navy 
HARDMAN  research  is  that  it  is  a  cross¬ 
domain  integrated  system.  The  manpower, 
personnel,  and  training  domains  are  inter¬ 
related.  If  an  analyst  makes  changes  in  any 
one  domain,  these  changes  will  most  likely 
effect  the  other  two  domains.  For  example, 
if  you  reduce  the  number  of  people  you  plan 
to  use  to  perform  maintenance  on  a  new 
fighter,  you  then  have  to  expand  the  job 
definitions  of  the  remaining  people  to  cover 
those  tasks  that  would  have  been  performed 
by  the  manpower  spaces  that  were  elimi¬ 
nated.  Once  you  have  expanded  the  job 
descriptions  for  the  remaining  people,  your 
training  program  becomes  longer  and  more 
complex  for  both  initial  and  recurring 
training.  Unfortunately,  the  domains  are 
managed  separately,  and  the  reality  is  that 
when  changes  are  proposed  within  any  one 
domain  (e.g.,  the  manpower  community), 
those  changes  may  be  coordinated  with  Ae 
other  domains  (personnel  and  training)  but 
there  has  been  no  mechanism  available  to 
study  the  long  term  impacts  of  these 
changes.  The  MPT  DSS  will  include  func¬ 
tional  relationships  within  the  integrated 
system  tradeoff  analysis  methods  to  auto¬ 
matically  reflect  the  horizontal  cross-domain 
effects  of  making  changes  in  one  functional 
domain.  This  cross-domain  capability  will 
greatly  enhance  the  ability  to  demonstrate 
the  cost  impacts  associated  with  different 
policy  decisions  in  any  one  single  domain. 

The  prototype  MPT  DSS  will  focus  on 
supporting  the  MPT  analysis  of  Air  Force 
aircraft  systems  but  will  be  designed  so  that 
it  can  be  applied  to  any  type  of  system. 
Application  to  systems  other  than  Air  Force 
aircraft  systems  will  require  analysts  to 
expand  existing  library  files.  The  MPT  DSS 
will  concentrate  on  assessing  MPT  require¬ 
ments  for  the  maintainers  and  support 
personnel  who  work  directly  on  the  system 
in  the  operational  units  in  which  the  weapon 
system  will  be  fielded.  More  specifically. 


task-level  MPT  analyses  will  be  conducted 
on  maintainers  and  the  support  personnel 
whose  workload  is  directly  driven  by  the 
system.  Operator  crew  size  will  be  an  input 
to  the  MPT  DSS.  Total  manpower  for 
operators,  training  personnel,  and  support 
personnel  whose  workload  is  not  directly 
driven  by  the  system  will  be  determined  by 
Air  Force  manpower  standards  that  ded 
with  aggregate  workload,  not  individual 
tasks.  The  MPT  DSS  will  contain  both 
existing  and  new  analytical  tools. 

The  MPT  DSS  system  consists  of  three 
major  components:  a  System  Development 
Subsystem  (SDS),  a  Data  Base  Integration 
Subsystem,  and  the  Analysis  Tools 
Subsystem. 

MPT  DSS  Software  Components 

When  conducting  an  MPT  analysis,  select¬ 
ing  the  BCS  is  a  significant  first  step.  The 
SDS  will  assist  MPT  analysts  in  cf^nstruct- 
ing  the  BCS,  populating  the  BCS  task-level 
data  bases  with  appropriate  government  and 
contractor-furnished  data,  and  maintaining 
and  updating  the  BCS  data  throughout  the 
acquisition  process.  The  SDS  methodology 
includes  techniques  to  match  new  system 
functional,  performance,  and  design  charac¬ 
teristics  to  those  of  existing  Air  Force 
equipment,  at  appropriate  levels  of  system 
indenture. 

An  integrated  MPT  data  base  is  needed  to 
support  the  MPT  DSS.  The  system  must  be 
capable  of  extracting  and  integrating  MPT 
data  from  external  Air  Force  data  sources  in 
a  user-fnendly  manner.  The  Data  Base 
Integration  Subsystem  will  help  Air  Force 
MPT  analysts  obtain  and  use  the  input  data 
needed  for  an  MPT  DSS  application.  The 
subsystem  will  request,  extract,  and  process 
data  from  external  sources;  integrate  input 
data  within  a  comprehensive  MPT  DSS  data 
architecture;  and  configure  the  data  to 
support  MPT  analyses  and  tradeoffs. 
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The  Analysis  Tools  Subsystem  attempts  to 
maximize  the  use  of  existing  tools  and 
techniques. 

System  Development  Subsystem  (SDS) 

The  SDS  component  consolidates  MPT 
related  predecessor  weapon  system  data  into 
a  BCS.  Then  as  design  informai  cn 
matures,  the  BCS  can  be  updated  to  form  a 
proposed  system  description.  The  predeces¬ 
sor  system  is  an  existing  system,  or  systems, 
that  have  components  or  missions  similar  to 
the  new  system  concept.  Descriptions  of 
predecessor  equipment,  maintainers  that 
repair  it,  manpower  standards  supporting  it, 
and  training  courses  related  to  it,  provide  a 
"footprint"  for  a  new  system.  As  identified 
in  Logistics  Support  Analysis  (LSA)  Task 
203  [12],  a  BCS  is  a  representative  system 
construct  composed  from  existing 
systems/subsystems  (predecessor  systems), 
support  systems,  and  lessons  learned  for 
performing  comparability  analysis.  The 
BCS  components  should  approximate  one  or 
more  of  the  new  system  functional, 
performance,  and  design  requirements.  As 
the  system  matures  and  actual  design  data 
become  available  through  the  MIL-STD- 
1388-2B  LSA  Record  (LSAR),  they  will 
replace  the  predecessor  system  data.  This 
will  permit  continual  improvement  of 
system  design  information,  and  provide 
better  predictions  of  Air  Force  MPT  costs 
and  support  requirements. 

Comparisons  between  the  BCS  and  the 
proposed  system  are  made  throughout  the 
acquisition  process  as  the  proposed  system 
design  evolves  and  design  alternatives  are 
considered.  Comparison  of  the  BCS  to  the 
proposed  system  requirements  in  the  early 
phases  of  the  acquisition  process  help  iden¬ 
tify  areas  of  technical  risk.  Comparison  of 
contractor  design  alternatives  to  the  BCS  in 
later  phases  also  help  identify  risk  areas 
(i.e.,  areas  for  which  the  contractor  is 
proposing  to  deliver  improvements  that  arc 


significantly  better  than  what  is  currently 
being  achieved). 

Data  Base  Integration 

This  component  accomplishes  two  tasks:  it 
links  geographically  separate  data  sotuces 
and  relates  data  between  dissimilar  data¬ 
bases. 

One  of  the  most  difficult  problems  for  an 
MPT  analyst  is  trying  to  obtain  all  of  the 
unrelated  data  from  locations  around  the 
country  to  support  the  integrated  analyses. 
This  burden  will  be  reduced  by  introducing 
a  system  that  will  automate  data  retrieval 
from  geographically  separate  data  sources. 
The  automated  system  will  allow  the  user  to 
check  a  block  on  the  user  screen  identifying 
what  data  are  required.  Then,  through  over¬ 
night  unattended  file  transfer  over  the 
Defense  Data  Network  (DDI-J)  or  by  modem 
connection  for  direct  attended  retrievals,  the 
data  will  be  electronically  gathered  to  the 
analyst’s  machine. 

Anotlier  major  challenge  is  the  process  of 
relating  weapon  system  specific  data 
(weapon  system  specific  job  task  lists)  to 
occupational  data  (which  Air  Force 
Specialties  (AFSs)  accomplish  those  tasks). 
In  the  past,  this  process  was  accomplished 
by  gathering  a  group  of  subject  matter 
experts  (SMEs)  and  having  them  laboriously 
relate  the  data.  Earlier  research  [13]  showed 
that  we  are  able  to  automate  the  process 
through  a  semantic  analysis  process  with 
about  an  80  per  cent  text  match.  The  SME 
time  required  is  reduced  by  about  two 
thirds. 

This  Database  Integration  Subsystem  will 
provide  the  data  needed  for  the  BCS  library 
and  the  suite  of  analysis  tools.  Maintenance, 
occupational,  personnel,  and  logistic  data 
from  current  systems  will  be  used.  Once 
predecessor  systems  are  identified,  the 
appropriate  task-level  data  will  be  extracted. 
TTie  Maintenance  Data  Collection  System 
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(MDCS),  Core  Automated  Maintenance 
System  (CAMS),  Logistic  Composite  Model 
(LCOM),  Occupational  Research  Data  Bank 
(ORDB),  APR  173-13  -  Air  Force  Cost  and 
Planning  Factors,  and  Logistic  Support 
Analysis  Records  (LSAR)  files  will  be  used. 
Such  data  bases  will  have  their  task-level 
data  linked  and  extracted.  It  data  for  a 
specific  sub-system  are  unavailable,  then 
SMEs  will  be  used.  Part  of  this  effort  will 
require  precise  definitions  of  tasks  and 
comparisons  between  the  actual  task 
statements. 

Information  about  how  to  investigate  the 
data  that  are  available  fi'om  various  sources 
will  be  provided  to  the  MPT  analyst  in  the 
form  of  help  screens.  The  generic  content 
and  structure  of  data  within  each  source  will 
be  described  including  the  level  of  equip¬ 
ment  indenture  for  which  data  are  usually 
available.  For  data  sources  hosted  at  a 
single,  or  a  few  geographic  locations,  the 
help  screens  will  include  contact  points  to 
whom  data  inquiries  or  requests  may  be 
directed. 

Data  Base  Architecture  -  After  task  and 
MPT  data  are  extracted  from  external 
sources,  they  must  be  incorporated  into  a 
common  data  architecture  within  the  MPT 
DSS.  The  data  architecture  design  will 
depend  on  the  data  meanings  and  relation¬ 
ships  of  the  data  extracted  from  the  external 
source  data  bases.  A  different  data  element 
will  be  established  in  the  MPT  DSS  data 
architecture  for  every  unique  data  element 
"meaning"  imported  into  the  DSS.  This  will 
accommodate  the  several  meanings  expected 
for  data  elements  such  as  task,  failure  rate, 
etc.  If  task  means  training  task  in  one  data 
base,  maintenance  task  in  another,  and 
occupational  survey  task  in  still  another, 
these  tasks  will  be  defined  as  separate  data 
elements.  The  MPT  DSS  data  architecture 
will  incorporate  standard  MPT  data 
elements  defined  by  efforts  such  as  the  DoD 
Computer-aided  Acquisition  and  Logistics 
Support  (CALS)  program. 


The  MPT  DSS  data  architecture  will  ensure 
that  relationships  between  data  elements 
within  a  given  source  data  base  will  be 
preserved  within  any  data  extracted  from 
that  source.  This  will  provide  the  capability 
to  effectively  relate  key  MPT  analysis  find¬ 
ings  and  tradeoff  results  to  the  processes  that 
generate  the  source  data.  For  example,  the 
LSA  control  number  (LCN)  and  task  code 
from  the  LSAR  are  not  expected  to  be 
primary  keys  within  the  MPT  DSS.  How¬ 
ever,  by  preserving  the  relationship  of 
LSAR  task  data  to  their  LCNs  and  task 
codes,  any  system  design  recommendations 
resulting  from  MPT  DSS  analyses  and 
tradeoffs  can  be  communicated  to  system 
designers  and  logisticians  using  familiar 
LCN/task  code  keys. 

Analysis  Tools  Subsystem 

This  component  is  the  core  of  the  MPT 
DSS.  Once  the  data  are  available,  it  must  be 
analyzed  and  examined.  The  integrated  set 
of  analysis  tools  will  be  designed  to  support 
a  step-wise  process  model  for  forecasting 
requirements  based  on  best  information 
available.  This  subsystem  includes  seven 
analysis  methodologies,  two  tradeoff  tech¬ 
niques,  two  analysis  aids,  and  a  planning 
aid. 

The  analysis  tools  include  a  Specialty 
Structuring  Tool  to  structure  jobs  from  the 
ground  up,  at  the  task  or  task  cluster  level  or 
restructure  a  specialty  starting  from  an  exist¬ 
ing  definition;  a  Personnel  Aptitude  and 
Characteristics  model  to  ensure  that  the 
collection  of  job  tasks  does  not  require 
unreasonably  high  aptitude  levels  or  physi¬ 
cal  profile  characteristics  that  can't  be 
supported  by  the  current  or  future  Air  Force 
population;  a  Training  Resources  and 
Requirements  Tool  to  project  an  estimate  of 
resources  needed  to  establish  and  maintain 
the  training  pipeline;  Manpower  Estimates 
Tool  to  determine  the  number  of  people 
required  to  operate,  maintain,  support,  and 
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train  a  single  unit  or  squadron;  a  Force 
Structuring  Tool  to  aggregate  the  manpower 
estimates  into  wings,  groups,  MAJCOMS, 
and  force  level  projections  using  approved 
overhead  and  support  ratios  (sufficient  to 
support  manpower  estimate  reports  required 
by  DoD);  an  Inventory  Projection/Civilian 
Availability  Model  to  determine  whether  the 
civilian  populace  can  support  the  level  of 
aptitude  in  the  numbers  identified  from  the 
force  projection  model;  finally,  the  last 
model  is  a  LCC  model  that  will  present  a 
bottom-line  dollar  figure  to  show  the  MPT 
related  costs  of  the  system. 

The  individual  methodologies  briefly 
described  above  are  useful,  but  more  impor¬ 
tant  are  the  techniques  to  permit  interaction 
among  the  individual  tools  to  tradeoff  the 
manpower,  personnel,  and  training  domains. 
There  is  a  great  deal  of  interaction  between 
each  of  the  domains.  Significant  changes 
cannot  be  made  to  any  one  functional 
domain  without  affecting  the  other  two. 
Therefore,  fimctional  relationships  will  be 
included  which  describe,  in  analytic  terms, 
the  relations  between  the  various  M  and  P 
and  T  factors.  MPT  measures  of  effective¬ 
ness  (MOEs)  will  be  used  by  the  tradeoff 
process  to  provide  objective  criteria  - 
identifiable  and  explicit-  for  evaluating 
MPT  impacts  of  design,  operation,  and 
support  alternatives.  MPT  control  variables 
(i.e.,  the  variables  that  the  MPT  community 
controls,  and  can  change,  to  accommodate  a 
new  system)  will  be  identified  for  each  MPT 
MOE.  In  conducting  tradeoffs,  the  control 
variables  can  be  viewed  as  input  variables 
and  the  MPT  MOEs  can  be  viewed  as  the 
outcome  variables  that  are  used  to  assess 
MPT  impacts  for  all  types  of  tradeoffs 
(design,  support,  operations).  An  MPT 
Analysis  Tradeoff  Aid  will  identify  BCS  or 
proposed  system  high  (cost)  drivers  from  the 
MOEs.  Using  these  high  drivers,  an  analyst 
can  begin  conducting  a  sensitivity  analysis 
by  adjusting  the  control  variables 
contributing  to  the  MOE  identified  as  a  high 
driver.  The  MPT  Analysis  Aid  will 


empower  the  analyst  to  conduct  tradeoffs  in 
an  accelerated  mode  where  only  one  vari¬ 
able  has  been  changed  and  the  entire  analy¬ 
sis  (including  manpower  simulation)  will  be 
rerun,  or  in  detailed  mode  where  the  analyst 
can  make  multiple  changes  in  several  differ¬ 
ent  tools.  The  second  tradeoff  technique  is  a 
Comparison  Tool  that  will  let  an  analyst 
display  side-by-side  results  of  different 
completed  studies.  The  Comparison  Tool 
uses  summary  reports  and  graphics  to 
compare  differences  between  the  system, 
type,  and  versions.  The  system  refers  to  the 
type  of  proposed  system  you  are  analyzing, 
type  refers  to  the  type  of  analysis  you  are 
conducting,  and  version  refers  to  the  specific 
analysis  conducted.  By  varying  the  control 
parameters  you  create  multiple  versions  of 
the  analysis.  The  comparison  tool  allows 
you  to  compare  these  individual  versions. 
These  comparisons  can  demonstrate  the 
relative  value  of  different  MPT  approaches 
allowing  policy  options  or  system  design 
differences  to  be  studied.  An  analyst  can 
converge  on  an  optimal  solution  before 
beginning  the  full  documentation  needed  to 
complete  a  study  in  support  of  required 
acquisition  documentation. 

The  analysis  aids  are  tools  that  will  improve 
the  IMPACTS  analyst's  ability  to  use  the 
DSS.  An  integral  Navigation  Aid  assists  the 
user  in  correctly  using  the  integrated  analy¬ 
sis  methodology  for  different  types  of 
studies.  This  technique  consists  of  both  a 
navigation  aid  visually  depicting  the  steps 
necessary  to  complete  a  particular  type  of 
analysis  and  an  extensive  Context-Sensitive 
Help  component  providing  detailed  topic- 
related  assistance  throughout  all  stages  of 
the  analysis. 

The  plaiming  aid,  the  MPT  Pipeline  Tool. 
will  assist  Air  Force  analysts  in  scheduling 
the  MPT  resources  associated  with  deploy¬ 
ing  new  systems.  The  pipeline  tool  will 
consider  training,  organizational,  and 
support  pipelines,  to  ensure  plans  are 
developed  to  have  trained  people  where  and 
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when  they  are  needed  to  achieve  full  opera¬ 
tional  capability.  Outputs  from  this  tool 
include  a  master  milestone  chart  that  will  be 
a  PERT/CPM  chart  illustrating  the  time 
phasing  of  key  MPT  resourcing  events 
based  on  the  proposed  acquisition  strategy. 
The  tool  will  also  provide  system  training 
plan  information  and  a  forecast  of  required 
permanent  change  of  station  moves. 

Key  MPT-related  Acquisition  Products 
and  Processes 

Products  of  the  MPT  DSS  include  the 
Manpower  Estimate  Report;  Comparative 
Analysis  (LSA  task  203);  support  for  LSA 
tasks  303  -  Evaluation  of  Alternatives  and 
Tradeoff  Analysis,  401  -  Task  Analysis,  and 
402  -  Early  Fielding  Analysis.  The  MPT 
portions  of  a  Cost  and  Operational  Effec¬ 
tiveness  Analysis  (COEA)  can  be  prepared 
or  validated.  Finally,  the  DSS  will  produce 
analysis  summaries  and  Human  Systeins 
Integration  plans  in  a  format  needed  for  the 
IMPACTS  plan  and  Integrated  Program 
Summary 

MPT  DSS  Payoffs 

The  integrated  analysis  tool  and  decision 
support  system  will  assist  acquisition 
managers,  analysts,  and  MAJCOM  planners 
to  effectively  integrate  people  issues 
(numbers,  characteristics,  proficiency)  with 
equipment  (aircraft)  early  in  the  acquisition 
cycle.  Acquisition  specialists  can  use  the 
structured  analysis  approach  provided  by  the 
MPT  DSS  to  ensure  that  system  people  costs 
are  affordable,  jobs  are  properly  structured, 
and  people  are  trained  prior  to  the  system 
becoming  operational.  The  analysis  meth¬ 
odology  includes  cross-domain  effects  of 
different  weapon  system  designs,  logistics 
concepts,  or  occupational  and  organizational 
structures.  Different  policy  decisions  can  be 
modeled  and  the  cost  impart  of  those 
decisions  identified. 


Program  Test  and  Evaluation 

The  final  program  phase  will  be  a  compre¬ 
hensive  test  and  evaluation  of  the  whole 
prototype  system.  Development  of  the  MPT 
DSS  will  involve  a  prototype  demonstration 
of  a  specific  weapon  system.  This  will 
permit  the  incorporating  actual  data  and 
requirements  in  the  MPT  DSS  and 
identifying  issues  for  resolution.  A  rigorous 
test  of  the  final  integrated  MPT  DSS  proto¬ 
type  will  be  conducted  under  stringent  stan¬ 
dards  using  still  another  weapon  system 
with  intent  to  validate  the  overall  process 
model.  Close  cooperation  with  the  SPO  and 
the  manpower,  personnel,  training  and  using 
communities,  especially  MPT  planning 
teams,  will  be  a  key  part  of  the  DSS  devel¬ 
opment  and  validation.  Incremental  deliv¬ 
ery  of  the  MPT  DSS  is  planned  as  each 
subsystem  is  completed. 

Hardware  and  Software 

The  system  will  operate  on  an  80486  class 
(or  better)  microcomputer  in  a  Microsoft 
Windows  environment.  Anticipated  hard¬ 
ware  requirements  include  1  GB  of  data 
storage  and  16  MB  of  RAM.  The  software 
will  be  written  in  the  C-h-  programming 
language  and  follow  object  oriented  design 
constructs.  There  will  be  full  document¬ 
ation  of  the  entire  system,  including  user's 
manuals,  design  dociaments,  technical 
reports,  and  detailed  system  and  software 
specifications. 

Summary 

The  MPT  DSS  is  intended  to  help 
MAJCOM  requirements  planners  and  SPO 
analysts  and  decision  makers  insert  MPT 
factors  into  the  acquisition  process.  It  will 
provide,  in  one  integrated  system,  a  means 
of  accessing  task-level  data,  analyzing  it, 
and  presenting  it  for  review  and  study  at  any 
milestone  in  the  acquisition  process.  This 
type  of  tool  has  long  been  needed  by  the 
MPT  commrmity  and  SPO  decision  makers. 
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ABSTRACT 

The  planning  for  training  systems  has  gained  increased  emphasis  in  the  Department  of 
Defense  (DoD)  community  as  training  devices  have  become  more  complicated  and 
expensive.  DoD  Instruction  (DoDI)  5000.2,  "Defense  Acquisition  Management  Policies 
and  Procedures"  dated  February  23,  1991,  directs  the  evolution  of  a  "training  system" 
concept  concurrently  with  the  development  of  new  weapon  systems.  Draft  DoD  Directive 
1430. 13,  Development  and  Life-^cle  Management  of  Milit^  T raining  Systems,  directs 
a  structured  approach  in  this  training  system  concept  formulation.  The  directive  requires 
that  the  developing  agency  thoroughly  analyze  the  intended  use,  affordability,  and 
effectiveness  of  the  new  training  concept.  To  acquire  a  training  system  in  this  framework 
of  a  systematic  training  analysis  process,  DoDD  1430. 13  requires  that  associated  hardware, 
courseware,  and  implementing  personnel  be  studied  in  the  context  of  an  economic 
analysis,  cost  effectiveness  analysis,  and  training  effectiveness  analysis. 


In  the  implementation  of  the  Advanced  Assault  Amphibian  Vehicle  (AAAV)  Training 
System  Plan  as  directed  by  DoDl  5000.2,  Marine  Corps  planners  will  be  customizing  the 
requirements  of  M1L-STD-1379D,  Military  Training  Programs,  under  the  framework  of 
OPNAVINST  1500.59,  Surface  Warfare  Training  System  Acquisition  Process  and 
Responsibilities.  Program  planners  have  found  that  by  combining  these  two  training  and 
acquisition  oriented  approaches,  a  thorough  "training  systems"  approach  emerges  which  is 
fully  consistent  with  the  intent  of  DODI  5000.2.  This  paper  outlines  the  approach  with  the 
intent  that  it  can  benefit  the  planning  of  other  military  program  managers. 


INTRODUCTION 

To  date,  the  planning  for  major  military 
training  procurements  has  of  necessity  been 
driven  by  the  engineering  hardware 
complexity  and  the  dollar  value  of  the 
procurement.  For  instance,  in  the 
procurement  of  a  complex  weaptrn  systems 
trainer.  Marine  Corps  program  managers 
have  been  able  to  utilize  OPNAVINST 
1500.59,  Surf  ace  Warf  are  Training  System 
Acquisition  Process  and  Responsibilities, 
to  organize  and  schedule  associated 
engineering,  fiscal,  logistics,  and 
manpower  requirements.  However,  when 
the  training  procurement  was  geared  to  a 
classroom  "training  system"  such  as 


computer-based  instruction  (CBl), 
interactive  video  disc  (IVD),  or  interactive 
courseware  (ICW),  the  acquisition  was 
driven  more  by  courseware  and  training 
issues.  In  the  latter  case,  MIL-STD-1379, 
Military  Training  Programs,  has 
emphasized  such  training  issues  as  training 
situation  analysis,  lesson  specification 
development,  and  training  evaluation.  The 
strength  of  the  hardware  acquisition  driven 
approach  \s  a  thorough  definition  of  the 
engineering  complexity  and  logistics  of  the 
required  training  hardware.  The  strength  of 
the  training  analysis  and  courseweuc 
driven  approach  is  a  thorough  definition  of 
the  training  requirement  and  "intended  use" 
by  the  school  or  training  command. 
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TRAINING  SYSTEM 
DEVELOPMENT  APPROACH 


TASK  SECTION  100.  TRAINING  PROGRAM 
MANAGEMENT 


This  plan  was  developed  after  a  review  of 
training  systems  developed  within  the 
Marine  Corps  and  the  Navy.  Since  the 
Advanced  Assault  Amphibian  Vehicle 
(AAAV)  will  be  a  new  weapon  system,  it 
was  assumed  that  the  new  training  system 
should  build  upon  the  strengths  of  both 
Marine  Corps  and  Navy  training  practices. 
Past  training  program  "lessons  learned" 
were  factored  into  the  proposed  training 
development  approach.  The  Navy 
HARDMAN  pixjcess  has  been  utilized  as  a 
major  planning  tool  as  well  as  the  planned 
development  of  a  AAAV  Marine  Corps 
Training  Plan  (MCfP).  This  training 
system  plan  will  outline  the  activities 
necessary  to  build  the  AAAV  training 
system.  The  AAAV  MCTP  will  identify 
those  activities  necessary  to  ensure  the 
ongoing  management  of  the  AAAV  training 
system. 

Overview.  As  the  AAAV  weapon  system 
design  matures  during  the  early  acquisition 
phases,  more  and  more  detail  on  training 
system  requirements  will  become  available. 
The  training  system  program  goal  is  to  use 
weapon  system  design  information  early 
enough  in  the  development  process  to 
ensure  the  deployment  of  the  AAAV 
training  system  before  the  deployment  of 
the  weapon  system.  A  listing  of  the  tasks 
identified  by  MIL-STD-1379D  for  the 
systematic  development  of  training 
programs  is  presented  in  'fable  1. 

The  MIL-STD-1379Drr«£>ii>ij?o«<I 
courseware  oriented  development  tasks  are 
identified  by  their  rcquirments  for  each 
weapon  system  pha.se  in  f  igure  1,  Figure 
1  also  presents  training  hardware  oriented 
acquisition  tasks  (ie.,  (JPNAVTNST 
1500.59.  guidance)  for  each  weapon 
system  phase,  'fhe  guidance  from  these 
two  documents  (MIL-STD-  1.379D  & 
OPNAVINST  1500.59)  has  been  integrated 
as  the  basis  for  this  AAAV  'f raining  System 
Plan.  The  following  sections  of  this  plan 
will  describe  the  proposed  AAAV  training 
system  development  approach. 


Task  101:  Training  Situation  Analysis 
Task  102;  Training  Program  Development  & 
Management  Planning 
Task  103:  Training  Development  Control 
Task  104:  Training  Development  Requirments 
Identification 

Task  105:  Training  Implementation  Hanning 
Task  106:  Training  Implementation  Control 
Task  107:  Training  System  Support  Materials 

TASK  SECnON  200:  TRAINING 
PRtXlRAM  ANALYSIS  AND  DESIGN 

Task  201:  Mission,  Collective,  Individual  & 

Occupational  Training  Task  Analysis 
Task  202:  Training  Technology  Assessment 
Task  203:  Learning  Analysis 
Task  204:  Media  Selection 
Talcs  205:  Tests  for  Measurement  of  Personnel 
Achievement 

Task  206:  Training  System  Alternatives 
Identification 

Task  207:  Training  System  Modification 
Requirements 

Task  208:  Training  System  Functional 
Requirments 

Task  209;  Training  System  Facilities 
Requirments  Identification 
Task  210:  Lesson  Specification  Development 
Task  211:  Instructional  Media  Design 

TASK  SECTION  .300:  TRAINING 
MATER1AI..S  DEVELOPMENT 

l'ask301:  Instructional  Media  Production 
Task  302:  Training  System  Support  Materials 
Task  303:  Training  Materials  for  Instructors 
Task  304:  Training  Materials  for  Trainees 

TASK  SECTION  400:  TRAINING 
SUPPORT,  CONDUCT  AND  ADVISORY 
SERVICES 

Task  401:  Conduct  of  Training 
Task  402:  Training  Evaluation 


Table  1.  Requirements  for  Military 
Training  Programs  (MIL-STD- I379D) 
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Figure  1.  Traiaiag  Sjstaa.  AcQuiaitioa  H^qu ireHtents  bj  Flias« 
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Development  Approach.  Training 
program  tasks  identified  by  I379D  will  be 
selectively  applied  for  the  A  AAV  Training 
System  as  the  AAAV  weapon  system 
evolves.  These  training  program  tasks  will 
be  implemented  within  the  context  of  the 
training  system  planning  and  acquisition 
requirements  of  OPNAVINST  1500.59. 
The  tasks  will  be  tailored  during  each 
weapon  system  phase  depending  on  the 
emerging  training  needs  of  the  program. 

CONCEPT  EXPLORATION 
PHASE  R EQUIPMENTS 

HARDMAN  Analysis.  HARDMAN 
analysis  has  been  conducted  to  establish 
baseline  projections  for  AAAV  manpower, 
personnel,  and  training  (MPl  ) 
requirements.  Several  training  situation 
analyses,  Task  101,  were  completed 
during  the  prcconcept  phase  as  input  to  the 
HARDMAN  analysis  The  following  three 
reports  were  required  from  each  industry 
contractor  conducting  the  AAAV  concept 
design  study,  H.ARDMAN  .Assessment 
Report;  Manpower,  Personnel,  and 
'fraining  (Concept  I>x^ument  (MPT  CD);  and 
Manpower,  Personnel,  and  Training 
Resource  Requirements  lXx:ument 
(MPTRRD). 

The  baseline  projection  will  be  updated 
during  the  demonstration/validation  phase 
as  the  AAAV'  dc.sign  evolves.  Based  on  the 
initial  H.ARDM.AN  analysis,  the  following 
requirements  have  been  identified  as  key 
areas  for  A  A.AV  training;  crew 
operations/ crew  coordination,  driver 
operations,  navigation,  communication, 
emergency  egress,  tunet  operations, 
weapons  training,  maintenance  training, 
emergency  maintenance  procedures, 
misvsion  planning,  tactics,  and  crew/infantry 
coordination. 

DEMDNSTR  ATION/VALIDATION 
PHASE  R EQUIPMENTS 

HARDMAN  Update,  fhe  AAAV  MPl 
re<]uirements  analysis  will  be  updated  as  the 
wca|x>n  system  design  evolves.  An  initial 


estimate  of  manpower  (lEM)  will  be 
developed.  The  lEM  will  include  an 
updated  projection  of  the  mtmpower 
required  to  operate,  maintain,  and  support 
the  weapon  system.  The  HARDMAN 
analysis  will  also  update  training  system 
requirements.  Training  and  training  device 
requirements  will  be  updated.  A  training 
situation  analysis  will  also  update  the 
organizational  requirements  at  the  unit  and 
institutional  levels. 

By  the  middle  of  the 
demonstration/validation  phase,  the 
weapon  system  design  will  have  evolved 
from  subsystem  to  equipment  to 
component  specification.  One  or  several 
AAAV  prototypes  will  be  developed. 

Based  on  this  increased  level  of  detail,  an 
updated  operator  and  maintainer  task 
analysis  (ie.,  Task  201)  will  be  developed. 
The  embedded  training  and  built-in-test 
capabilities  of  the  weapon  system  will  be 
completely  specified.  As  more  design 
detail  becomes  available,  the  AAAV  MPT 
projections  will  be  iteratively  updated. 

Marine  Corps  Training  Plan 
(MCTP).  The  updated  HARDMAN 
analysis  will  be  used  as  a  basis  for  (a)  the 
development  of  the  Marine  Corps 
equivalent  of  a  Navy  Training  Plan  (NTP) , 
and  (b)  the  implementation  of  training 
program  development  activities  specified  by 
MIL-STD- 1379.  The  MCTP  will  become 
the  primary  document  for  defining 
Manpower,  Personnel,  and  Training  (MIT) 
requirements  for  the  AAAV  including  the 
resources  (billets,  training  devices,  training 
equipment,  training  courses  and  military 
construction)  necessary  to  support  the 
weapon  system's  introduction  to  the  Marine 
Corps.  The  selection  of  MIL-STD- 1379 
training  program  tasks  will  ensure  that  the 
training  system  (courseweue  and  devices)  is 
developed  and  planned  to  systematically  fit 
the  requirements  and  constraints  of 
associated  schools,  units,  and  shipboard 
detachments. 

Military  Characteristics  (MCs). 
Military  Characteristics  (MCs)  for  training 
devices  will  be  developed  as  a  byproduct  of 
M1L-ST1>  1379  Task  208,  Training  System 
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Functional  Requirments.  The  MCs  will 
document  operational  and  performance 
requirements  as  a  basis  for  developing 
training  devices.  The  MCs  wilt  be  used 
key  documents  for  program  management 
review  and  as  source  documentation  for 
the  acquisition  of  the  AAAV  training 
system. 

Training  System  Acquisition  Plan. 

As  the  scope  of  the  training  system  is 
specified  through  the  training  front  end 
analysis,  acquisition  strategies  for  the 
training  system  will  be  developed  and 
finalized.  During  die 
demonstration/validation  phase,  a  Draft 
Training  System  Acquisition  Plan  will  be 
develop^  by  the  Naval  Training  Systems 
Center  for  review  by  the  DRPM  AAA. 

The  training  system  development  will  be 
separate  from  the  weapon  system 
development.  The  Naval  Training  Systems 
Center  has  been  designated  the  principle 
development  activity  (PDA)  for  the  AAAV 
Training  System.  A^uisition  approaches 
will  address  alternative  strategies  for  the 
development  of  training  devices, 
courseware,  classroom  instructional  media, 
and  mockups. 

MILCON  Plans/Submittais.  Based 
on  the  training  device  requirements 
specified  by  Ae  MCs,  site  surveys  from  the 
preconcept  phase  training  situation  analyses 
will  be  updated  A  T raining  Fxjuipment 
F-acilities  Retjuirements  (FIT^)  Plan  will  be 
developed  six  years  prior  to  the  ready-for- 
training  (RFT)  date.  The  EFR  plan  is  the 
basis  for  the  Training  System  Military 
Construction  (MILCON)  submittal. 

Military  construction  (MILCON)  plans  will 
be  coordinated  and  the  requisite 
documentation  will  be  submitted  based  on 
the  Training  System  EFR  Plan. 

POM  Submittals.  Program  and  budget 
resources  will  be  planned  for  the 
acquisition  and  life  cycle  support  of  the 
training  system.  POM  submittals  will  be 
programmed  annually  based  on 
requirements  identified  in  the  annual  MCTP 
review. 


FULL  SCA’  ^  DEVELOPMENT 
PHASE  REQUIRMENTS 

Maintenance  &  Logistics  Concepts. 
Maintenance  &  logistics  concepts  will  be 
developed  for  the  maintenance  and 
sparring  of  training  devices  and  the  update 
of  curricula,  courseware,  and  training 
devices.  An  ILS  program  will  be 
developed  for  the  training  system  to  achieve 
the  system  readiness  objectives  at  an 
affordable  life  cycle  cost.  Early  ILS 
program  activity  will  focus  on  designing 
affordable  support  characteristics  into  the 
training  system.  Subsequent  activity  will 
focus  on  acquisition,  evaluation  and 
deployment  of  support  resources.  The 
training  system  will  be  included  in  the 
Naval  Training  Systems  Center's  COG  "2* 
0  life  cycle  management  program. 

Marine  Corps  Training  Plan 
(MCTP)  Update.  During  full  scale 
development,  the  MCTP  will  be  updated  in 
coordination  completion  of 
MlL-STr>-1379  Task  105,  Training 
Implementation  Planning.  Institutional, 
unit,  and  onboard  training  plans  will  be 
more  detailed  given  the  completion  of  the 
initial  design  analysis. 


Training  System  Military 
Construction  (MILCON)  Initiated. 

Based  on  the  Training  Equipment  Facilities 
Requirements  (EFR)  Plan  developed  in  the 
previous  phase  the  military  construction 
(MIIGON)  for  training  facilities  will  be 
initiated.  The  implementation  will  ensure 
that  the  facilities  will  be  available  before 
delivery  of  the  training  system. 

Training  System  Acquisition 
Executed.  Based  on  the  acquisition 
planning  from  the  previous  phase,  the 
training  system  (training  devices,  training 
media,  and  courseware)  procurement/s  will 
be  initiated.  The  procurement  of  the 
training  system  will  have  the  same 
emphasis  as  the  procurement  of  the  AAAV 
to  ensure  that  the  training  system  delivery 
precedes  the  delivery  of  the  weapon 
system. 
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PRODUCTION  PHASE 
REQUIRMENTS 


DEPLOYMENT  PHASE 
REQUIRMENTS 


Training  Facility  Completed.  The 

training  racilitymodifications  and/or 
construction  will  be  completed  with 
appropriate  space  for  classroom, 
laboratory,  and  training  device  activities.  If 
af^ropriate,  the  coordination  of  satellite 
hookups  between  units  will  be  established 
for  battalion  level  distributed  training. 


Training  System  Developed.  The 
training  system  development  and 
implementation  will  be  guided  by  MIL- 
STD-1379Task  106,  Training 
Implementation  Control,  and  also  by  Task 
107,  Training  Evaluation  Planning.  A 
training  effectiveness  evaluation  plan 
(TEEP)  will  be  developed  to  measure  the 
performance  specified  by  the  initial  task 
analysis  conducted  during 
demonstration/validation. 

Ready  For  Training  (RFT). 

Acceptance  tests  will  be  conducted  on  the 
training  system  in  advance  of  the  RFT  date 
specified  by  the  Marine  Corps. 


Conflgnration  Management  Config¬ 
uration  management  d*  die  training  system 
will  be  built  into  the  POM  process  so  that 
the  training  system  reflects  the  evolving 
modifications  of  die  weapon  system. 
Training  system  updates  will  be 
incorporated  through  the  yearly  review  and 
revisions  of  the  AAAV  Marine  Corps 
Training  Plan. 


AAAV  Training  System  Milestones. 
Figure  2  presents  the  training  system 
schedule  and  milestones. 


Maintenance/Logistics  Concepts 
Implemented.  All  preparations  and 
contracting  for  the  life-cycle  support  of  the 
training  system  will  be  concluded  prior  to 
the  RFF  date.  The  support  of  the  Gaining 
system  will  be  initiated  concunrendy  with 
the  initiation  of  training. 
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SUMMARY  AND  CONCLUSIONS 

Currently  the  design  of  the  Advanced 
Assault  Amphibian  is  in  its  early  stages  and 
training  concepts  are  evolving  with  the 
competing  design  concepts.  DoD  emphasis 
on  baining  in  distributed  simulated 
environments  will  add  to  the  complexity  of 
the  training  analysis.  There  will  be  a  need 
to  understand  hardware  technology  issues 
such  as  interconection  requirments, 
standards,  and  protocols.  In  addition,  there 
will  be  a  ?  ced  to  fully  analyze  the  breadth 
and  depth  of  the  individual  and  collective 
training  requirment  as  well  as  proper 
feedback  and  evaluative  techniques  to 
ensure  training  with  the  minimum  of 
training  management  resource  penalties. 
This  training  system  picm  is  comprehensive 
enough  to  address  the  planning  and 
analysis  requirments  for  future  trmning 
hardware  and  courseware  mixes.  The  plan 
also  permits  the  program  manager  the 
flexability  to  fcKUs  and  refine  analayses  as 
the  training  system  requirments  become 
more  concrete  as  the  weapon  system  design 
evolves. 
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EVOLVING  CRITERIA  FOR  EVALUATION: 
THE  CHALLENGE  FOR  THE  INTERNATIONAL 
INTEGRATOR  OF  THE  90s. 


Joan  Fowler  &  Virgil  Gibson 
Security  Architects 
Grumman  Data  Systems 

Abstract 

This  paper  presents  a  comparison  of  three 
international  security  criteria  and  contrasts  their 
approaches.  It  demonstrates  the  integrator’s 
perspective  of  the  implications  for  designing  solutions 
to  meet  worldwide  information  protection  needs.  The 
implied  and  stated  use  of  each  of  the  criteria  is 
included  in  this  paper  with  a  description  of  the 
barriers  to  understanding  between  the  criteria.  A 
high  level  comparison  of  the  approaches  taken  in  the 
three  criteria  is  present  d.  From  the  system 
integrator’s  perspective,  the  causes  for  the  ambiguity 
between  these  criteria  are  discussed,  with  suggestions 
for  the  international  community. 

Introduction 

The  United  States  (U.S.)  Department  of 
Defense  (DoD)  published  in  1985  the  DoD  Trusted 
Computer  System  Evaluation  Criteria  (TCSEC)  [3] 
which  is  the  seminal  work  in  detailing  the  criteria 
which  guide  buyers  and  sellers  in  the  computer 
security  arena.  The  TCSEC,  and  its  interpretations; 
the  Trusted  Network  Interpretations  (TNI)  [7],  the 
Trusted  Database  Interpretation  (TDI)  [8],  and  the 
Computer  Security  Subsystem  Interpretation  (CSSI) 
(9],  form  the  nucleus  around  which  the  U.S.  Trusted 
Product  Evaluation  Program  has  developed. 

Recently,  two  new  criteria  have  appeared  on 
the  world  scene  which  are  to  be  used  in  the 
evaluation  of  security  applications  of  computer 
technology.  In  December  1990,  the  Canadian  System 
Security  Centre  (CSSC)  released  the  Canadian 
Trusted  Computer  Product  Evaluation  Criteria 
(CTCPEC),  Version  2.0.  The  CTCPEC  was  revised 
in  April,  1992,  as  '  ersion  3.0.  [2]  In  June  1991,  the 
Harmonized  Criteria  of  France,  Germany,  The 
Netherlands  and  the  United  Kingdom  (UK)  was 
published  as  the  Information  Technology  Security 
Evaluation  Criteria  (ITSEC),  Version  1.2  [5]. 


This  paper  is  divided  into  two  overall 
segments.  The  Erst  segment  is  a  description  of  the 
three  criteria:  their  intended  audiences,  life  cycle  use, 
and  approaches.  The  second  segment  describes  how 
the  international  systems  integrator  is  involved  in  the 
situation.  This  second  segment  also  provides  the 
system  integrator’s  perspective  of  the  first  segment: 
the  ambiguities  caused  by  the  three  international 
criteria,  and  suggestions  for  the  future. 

Stated  or  Implied  Use  of  Each  Criteria 

Before  discussing  the  intended  use  of  each  of 
the  criteria,  the  definition  of  a  couple  of  terms  is  in 
order.  For  the  purposes  of  this  paper,  a  product  "is 
a  hardware  and/or  software  package  that  can  be 
bought  off  the  shelf  and  incorporated  into  a  variety  of 
systems."  A  system  "is  a  specific  [data  processing] 
installation  v  n  particular  purpose  and  known 
operational  e  ironment’  [S]. 

Intended  Audience  for  the  Criteria 

All  three  of  the  criteria  have  a  common 
understanding  of  these  two  terms,  product  and 
system.  However,  the  three  criteria  are  not  intended 
to  be  used  in  the  same  context  with  regard  to  the  two 
terms. 

The  target  for  evaluations  using  the  U.S. 
TCSEC  has  historically  been  primarily  products.  Its 
focus  is  on  industry,  in  that  it  is  an  incentive  to 
vendors  to  develop  trusted  products.  An  entire  series 
of  guidelines  have  been  developed  to  aid  industry  in 
the  use  of  the  criteria.  The  TCSEC  is  intended  to 
provide  guidance  to  the  commercial  world  on  the 
development  of  trusted  products,  and  provide  a 
direction  for  the  growth  of  each  individual  product. 
Industry  may  choose  to  grow  their  product  toward 
more  security  features  or  more  assurance  in  these 
features. 

The  Canadian  CTCPEC  specifically  states 
that  the  target  for  its  evaluation  is  products. 
However,  the  focus  of  the  CTCPEC  is  on 
government.  The  criteria  does  not  provide  clear 
direction  for  the  growth  of  commercial  products.  It 
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provides  only  the  criteria  by  which  a  product  may  be 
evaluated  once  it  is  developed.  However,  the 
CTCPEC  very  conveniently  allows  a  vendor  to 
develop  a  product,  and  then  add  to  the  assurances  of 
the  product  by  improving  the  documentation  in  a 
growth  fashion. 

The  European  ITSEC  specifically  maintains 
that  its  target  for  evaluation  is  both  products  and 
systems.  The  ITSEC  states  that  "it  is  important  for 
the  sake  of  consistency  that  the  same  security  criteria 
are  used  for  both  products  and  systems;  it  will  then 
be  both  easier  and  cheaper  to  evaluate  systems 
containing  products  which  have  already  been 
successfully  evaluated.’  [5]  Because  of  the  loose 
nature  of  the  ITSEC  approach,  it  also  does  not 
present  a  clear  growth  direction  for  the  security 
features  of  products.  However,  like  the  CTCPEC, 
vendors  may  build  a  product/system  and  then  add  to 
the  assurances  of  the  product/system  by  improving 
the  documentation  and  receiving  a  higher  rating  for 
assurance. 

Barriers  to  Understanding 

There  are  further  barriers  to  understanding 
of  the  three  criteria.  The  use  of  terminology  in  the 
three  criteria  is  one  of  these  barriers.  Certain  actions 
in  the  overall  evaluation  process  are  not  universally 
identified  in  the  three  schemes.  To  illustrate  this 
point,  a  simile  of  the  process  by  which  a  household 
appliance  receives  the  Underwriters  Laboratories 
(UL)  Seal  of  Approval  will  be  used.  The  appliance 
is  developed  by  a  vendor  and  submitted  to  UL  for 
testing.  UL  initially  may  test  the  appliance  in  a 
laboratory.  This  is  the  same  as  the  evaluators 
scrutinizing  a  product  ’from  a  perspective  that 
excludes  consideration  of  a  specific  application 
environment.’  [2]  In  the  U.S.  and  Canada,  this 
proce.ss  is  referred  to  as  the  Evaluation  performed  by 
the  National  Computer  Security  Center  (NCSC)  and 
the  CSSC  respectively.  However,  since  the  ITSEC 
is  aimed  at  both  products  and  systems,  there  is  not  a 
clear  distinction  between  this  first  type  of  evaluation 
and  those  performed  for  specific  application 
environments. 

Returning  to  the  simile,  once  the  UL  has 
tested  an  appliance  in  the  laboratory,  it  may  be  tested 
for  application  to  some  specific  representative  types 
of  hou.seholds.  This  is  similar  to  the  ’assessment  to 
determine  whether  appropriate  security  measures  have 
been  taken  to  permit  the  system  to  be  used 
operationally  in  a  specific  environment.’  [3]  This 


second  type  of  evaluation  is  called  the  Certification 
evaluation  under  the  TCSEC.  In  the  CTCPEC,  the 
exact  same  type  of  evaluation  is  termed  a  Risk 
Assessment.  Again,  since  the  ITSEC  does  not 
differentiate  between  its  use  for  systems  and 
products,  it  only  recognizes  the  single  type  of 
evaluation. 

The  final  step  in  the  process  of  the  appliance 
testing  is  the  award  of  the  UL  Seal  of  Approval. 
This  is  awarded  after  the  evaluation;  and  an  official 
authorization  is  given  that  the  appliance  meets  the 
standards  for  the  Seal  of  Approval.  In  the  TCSEC, 
the  ’official  authorization  that  is  granted  to  an 
[Automatic  Data  Processing]  ADP  system  to  process 
sensitive  information  in  its  operational  environment* 
[3]  is  the  formal  approval/accreditation  procedure, 
usually  referred  to  as  an  accreditation.  The  CTCPEC 
refers  to  the  same  procedure  as  the  formal  approval. 
The  ITSEC  indicates  that  national  certification  bodies 
’will  award  certificates  to  confirm  the  rating  of  the 
security.*  [S]  In  the  ITSEC,  the  process  to  award 
these  certificates  is  referred  to  as  the  certification 
process. 

This  simile  of  household  appliance  UL 
testing  demonstrates  that  terminology  is  a  confusion 
factor  for  the  systems  integrator  using  all  or  a 
combination  of  the  three  criteria.  Since  evaluation, 
risk  assessment,  certification,  final  approval,  and 
accreditation  can  mean  different  processes  within  the 
approval  of  systems  and  products,  the  integrator  must 
be  aware  of  the  criteria  and  audience  when  using 
each  of  these  terms. 


Life  Cycle  Use  of  the  Criteria 

At  the  current  time,  the  practical  use  of  the 
TCSEC  is  much  more  specified  than  that  of  the  other 
two  criteria.  Since  the  TCSEC  has  been  available 
longer,  this  situation  is  not  surprising.  The  NCSC 
has  developed  guidelines  for  vendors  on  the  use  of 
the  TCSEC  for  product  evaluation  and  evaluation 
maintenance:  the  Trusted  Product  Evaluations,  A 
Guide  for  Vendors  [10];  and  the  Rating  Maintenance 
Phase,  Program  Document  [11].  The  first  describes 
the  procedure  which  the  vendor  should  use  to  take  a 
product  through  evaluation,  and  the  second  defines 
the  procedure  to  reevaluate  subsequent  releases  of 
evaluated  products. 

Neither  the  CTCPEC  nor  the  TCSEC  discuss 
the  practical  use  of  the  criteria.  The  ITSEC 
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acknowledges  that  there  needs  to  be  national 
certification  bodies  to  perform  evaluations  and  a 
national  procedure  for  the  maintenance  of  certified 
ratings  following  changes  to  an  evaluated  target. 
However,  the  details  of  both  of  these  procedures  are 
indicated  as  being  beyond  the  scope  of  the  ITSEC. 
Last  spring  the  Information  Technology  Security 
Evaluation  Manual  (ITSEM)  was  released  for 
comments  in  draft  version  0.2.  [6]  This  document  is 
meant  to  "harmonize  existing  security  evaluation 
methods  in  each  of  the  four  countries  in  order  to 
ensure  that  national  evaluation  methods  conform  to  a 
single  philosophy".  This  recent  development  is  a 
major  step  toward  standardization  of  the  use  of  the 
ITSEC.  The  latest  version  of  the  CTCPEC  indicates 
that  it  was  written  in  such  a  way  to  preclude  the  need 
for  interpretations.  However,  only  practice  will 
determine  the  procedures  for  the  use  of  that  criteria. 

Approach  Comparison 

The  schemes  defined  in  each  of  the  criteria 
are  different  in  some  ways,  and  the  same  in  others. 
However,  the  underlying  principle  is  common  to  the 
three  criteria.  This  underlying  principle  is  that  there 
are  security  features  (e.g.,  access  control,  auditing) 
which  are  required  to  be  available  in 
products/systems.  Additionally,  there  are  assurances 
(e.g.,  documentation,  testing)  which  must  be  used  to 
prove  that  the  features  are  performing  completely, 
correctly,  and  consistently.  All  three  of  the  criteria 
agree  on  these  concepts.  However,  the  packaging  of 
these  concepts  and  some  of  the  details  of 
implementation  of  these  principles  are  drastically 
different.  The  similarities  and  differences  are 
addressed  here  to  highlight  the  differences  which  will 
cause  a  barrier  to  international  acceptance  of 
products/systems  and  illustrate  the  knowledge  which 
an  international  integrator  needs  to  compete. 

The  U.S.  TCSEC  Approach 

The  TCSEC  has  specified  the  set  of 
functional  features  and  a  set  of  the  type  of  associated 
assurances  which  a  product  must  possess  for  each 
class  level.  These  features  and  assurances  are 
bundled  together  into  a  single  class.  There  are  four 
possible  divisions  containing  seven  classes  in  the 
TCSEC  scheme.  These  divisions  are  Minimal 
Protection  (D),  Discretionary  Protection  (Cl  and  C2), 
Mandatory  Protection  (Bl,  B2,  and  B3),  and  Verified 
Protection  (Al).  A  target  (product)  must  meet  all  of 
the  requirements  of  a  class  to  be  assigned  the  class 
level.  If  one  of  the  functions  or  assurances  of  a 


particular  class  is  not  available  for  the  target,  the  next 
lower  class  level  with  which  the  target  complies  is 
assigned  to  the  target. 

The  Canadian  CTCPEC  Approach 

The  CTCPEC  scheme  is  separated  into 
criteria  of  security  functionality  features  and 
assurances.  Unlike  the  TCSEC,  the  functional 
features  are  not  grouped  together  with  the  assurance 
requirements.  There  are  five  criteria: 
Confidentiality,  Integrity,  Availability, 
Accountability,  and  Assurance.  Each  criteria  is 
further  divided  into  divisions  and  levels.  The 
Confidentiality  criteria  is  decomposed  into:  covert 
chatmels  (CC-0,  CC-1,  CC-2,  and  CC-3); 

discretionary  confidentiality  (CD-0,  CD-I,  CD-2, 
CD-3,  and  CD-4);  mandatory  confidentiality  (CM-0, 
CM-1,  CM-2,  CM-3,  and  CM-4);  and  object  reuse 
(CR-0  and  CR-1).  The  Integrity  criteria  is  divided 
into:  discretionary  integrity  (ID-0,  ID-1,  ID-2,  ID-3, 
and  lD-4);  mandatory  integrity  (IM-0,  IM-1,  IM-2, 
lM-3,  and  lM-4);  physical  integrity  (lP-0,  IP-1,  IP-2, 
lP-3,  and  lP-4);  rollback  (lR-0,  IR-1,  and  lR-2); 
separation  of  duties  (lS-0,  lS-1,  lS-2,  and  lS-3)  and 
self  testing  (lT-0,  IT-1,  IT-2,  and  IT-3).  The 
Availability  criteria  is  divided  into:  containment 
(AC-0,  AC-1,  AC-2,  and  AC-3):  robustness  (AR-0, 
AR-1,  AR-2,  and  AR-3);  and  recovery  (AY-0,  AY-1, 
AY-2,  and  AY-3).  The  Accountability  criteria  is 
partitioned  into:  identification  and  authentication  (WI- 
0,  WI-1,  and  Wl-2);  audit  (WA-0,  WA-1,  WA-2, 
WA-3,  and  WA-4);  and  trusted  path  (WT-0,  WT-1, 
and  WT-2).  Finally,  the  Assurance  criteria  consists 
of  trust  (T-0,  T-1,  T-2,  T-3,  T-4,  T-5,  T-6,  and  T- 
7).  Tables  1  through  4  illustrates  the  CTCPEC 
profiles  which  correspond  to  the  TCSEC  Classes  C2 
through  B3  respectively.  There  is  no  correlation  to 
these  profiles  and  the  associated  levels  of  trust  within 
the  corresponding  TCSEC  Classes.  An  equivalent 
TCSEC  profile  does  not  imply  that  a  TCSEC  rating 
meets  the  profile.  Hence,  these  are  to  be  considered 
as  one  way  mappings. 

As  in  the  TCSEC,  a  target  (product)  must 
meet  all  of  the  requirements  for  a  specific  level, 
otherwise  it  is  assigned  the  lowest  class  with  which 
the  target  complies  completely.  Each  criteria  division 
contains  a  level  designated  "0".  This  level  is 
reserved  to  targets  which  have  been  evaluated  but 
failed  to  meet  the  requirements  of  any  of  the  higher 
levels  for  the  category  division. 


195 


TaMe  1.  CTCPEC  Profile  Equivalent  to 
TCSEC  Class  C2 


Functionality 

Division/Mechanism 

Level 

Confidentiality 

Discretionary 

2 

Object  Reuse 

1 

Integrity 

Discretionary 

1 

Separation  of  Duties 

1 

Self  Testing 

1 

Accountability 

I& A 

1 

Audit  Level 

1 

The  European  ITSEC  Approach 

The  ITSEC  also  has  separated  the 
functionality  into  a  separate  rating  from  the 
assurances  or,  as  stated  in  the  ITSEC,  the 
effectiveness  and  correctness  aspects  of  assurance. 
There  are  ten  example  functionality  classes.  The  first 
five  are  closely  tied  to  the  TCSEC  classes  (F-Cl,  F- 
C2,  F-Bl,  F-B2,  and  F-B3).  Table  5  maps  the 
ITSEC  classes  to  the  TCSEC  classes.  This  mapping 
is  a  general  guide,  the  two  criteria  schemes  do  not 
directly  correspond  to  each  other. 

The  other  five  classes  are  new  in  the  ITSEC: 
high  integrity  requirements  (F-IN);  high  requirements 
for  availability  of  complete  or  special  functions  of  the 
target  (F-AV);  high  requirements  for  data 
communication  integrity  (F-DI);  high  demands  for 
data  communication  confidentiality  (F-DC);  and 
networks  with  high  demands  on  confidentiality  and 
integrity  of  information  (F-DX).  These  classes  are 
examples  and  not  obligatory.  They  can  only  be  used 
if  the  target  (product  or  system)  contains  all  aspects 
of  the  class.  A  target  may  reference  one  or  more  of 
these  example  functionality  classes  to  define  part  or 
all  of  its  fimctions.  As  an  alternative  to  the  use  of 
the  example  pre-defined  functionality  classes,  the 
sponsor  of  evaluation  can  specify  the  security 
enforcing  functions  of  the  target. 


Table  2.  CTCPEC  Profile  Equivalent  to 
TCSEC  Class  B1 


Functionality 

Division/Mechanism 

Level 

Confidentiality 

Discretionary 

2 

Mandatory 

2 

Object  Reuse 

1 

Integrity 

Discretionary  or 
Mandatory 

1 

Separation  of  Duties 

1 

Self  Testing 

1 

Accountability 

I&  A 

1 

Audit  Level 

1 

correctness,  an  assessment  of  effectiveness  based  on 
a  vulnerability  analysis  of  the  target  is  performed. 
There  is  a  pass/fail  designation  of  the  evaluation  on 
effectiveness  grounds. 


Table  3.  CTCPEC  Profile  Equivalent  to 
TCSEC  Class  B2 


Functionality 

Division/Mechanism 

Level 

Confidentiality 

Covert  Chaimels 

1 

Discretionary 

2 

Mandatory 

3 

Object  Reuse 

1 

Integrity 

Discretionary  or 
Mandatory 

1 

Separation  of  Duties 

2 

Self  Testing 

I 

Accountability 

I&  A 

1 

Audit  Level 

1 

Trusted  Path 

1 

There  are  seven  possible  correctness  levels 
(EO,  El,  E2,  E3,  E4,  E5,  and  E6)  described  in  the 
ITSEC.  In  addition,  following  the  evaluation  of  the 
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Table  4.  CTCPEC  Profile  Equivalent  to 
TCSEC  Class  B3 


Functionality 

Division/Mechanism 

Level 

Confidentiality 

Covert  Channels 

1 

Discretionary 

3 

Mandatory 

3 

Object  Reuse 

1 

Integrity 

Discretionary  or 
Mandatory 

1 

Separation  of  Duties 

2 

Self  Testing 

I 

Availability 

Recovery 

1 

Accountability 

I& A 

1 

Audit  Level 

2 

Trusted  Path 

2 

The  ITSEC  approach  is  more  flexible,  and 
more  open  to  interpretation  by  all  of  the  national 
certification  bodies  which  will  perform  the 
evaluations.  This  flexibility  may  limit  the  ability  for 
any  future  reciprocal  recognition  of  certifications. 
The  actual  practice  in  the  use  of  the  criteria  in  the 
future  will  determine  the  feasibility  of  this  approach. 
For  the  systems  integrator,  this  approach  ha.s  the 
potential  to  evolve  into  four  different  practical 
usages,  one  for  each  of  the  involved  countries: 
United  Kingdom,  The  Netherlands,  France  and 
Germany. 

Use  of  the  Three  Criteria 

For  an  example  of  the  rating  of  a  product 
under  each  of  the  schemes,  we  selected  a  fictional 
product  that  is  an  M-Component  (Mandatory  Access 
Control)  under  the  TNI  with  a  rating  of  TCSEC 
Class  B1  M-Component.  The  TNI  "allows  for  the 
evaluation  of  components  which  in  and  of  themselves 
do  not  support  all  the  policies  required  by  the 
TCSEC"  but  which  can  be  reused  "in  different 
networks  without  the  need  for  a  re-evaluation  of  the 
component."  [7] 


This  same  product  when  rated  under  the 
CTCPEC  would  have  a  rating  ofCM-1,  CR-1,  WI-3, 
and  T-3.  This  designation  is  a  clear  correspondence 
between  the  two  criteria,  TCSEC  (TNI)  and  the 
CTCPEC.  The  correspondence  is  not  as  clear  to  the 
nSEC  scheme.  The  closest  ITSEC  example  rating 
is  F-Bl  and  E3.  However,  a  F-Bl  has  requirements 
which  do  not  correlate  to  the  M-Component 
designation  of  the  TNI  (e.g..  Identification  and 
Authentication,  "A.20  The  TOE  shall  uniquely 
identify  and  authenticate  users"  [S]).  Further,  there 
are  requirements  for  F-Bl  which  are  not  designated 
in  the  U.S.  scheme  for  a  B1  M-Conqranent, 
reference  [1]  page  48. 

Approach  Conclusion 

This  discussion  of  the  differing 
terminologies,  requirements,  and  approaches  of  the 
three  criteria  must  lead  to  the  conclusion  that  there  is 
no  consistency  between  them.  This  inconsistency 
leads  to  confusion  in  the  systems  integration 
community  among  others.  The  potential  effects  that 
this  confusion  will  have  on  this  community  are 
discussed  below. 


Table  5.  ITSEC  to  TCSEC  Mappings 


TCSEC  Class 

ITSEC  Class 

D 

EO 

Cl 

F-Cl,  El 

C2 

F-C2,  E2 

B1 

F-Bl,  E3 

B2 

F-B2,  E4 

B3 

F-B3,  E5 

A1 

F-B3,  E6 

Systems  Integrator  Perspective 

Systems  vendors  and  integrators,  who  expect 
to  survive  through  the  decade  of  the  90s,  will  have  to 
contend  with  the  trusted  systems  criteria  discussed 
above.  Their  differing  requirements  and  approaches, 
and  the  implications  of  their  use  will  determine 
international  competitiveness. 
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What  is  a  Systems  Integrator? 

In  this  paper,  the  term  System  Integrator  is 
defined  as  follows:  A  systems  integrator  provides  the 
expertise  to  cost  effectively  bring  together  divergent 
products  from  multiple  product  lines  to  solve  a 
specific  operational  problem  in  a  specific  installation. 
In  the  case  where  the  problem  includes  protection  of 
information,  some  of  the  products  will  have  security 
functionality,  and  will  likely  have  been  evaluated  by 
one  of  these  international  schemes. 

Systems  integrators,  as  defined  for  this 
paper,  typically  respond  to  Requests  for  Proposals 
(RFPs)  and  Invitation  for  Bids  (IFBs)  from 
Governments  and  related  Organizations,  such  as  the 
North  Atlantic  Treaty  Organization  (NATO),  solving 
a  specific  problem  (part  of  which  is  assumed  to 
include  security)  in  a  given  environment. 
Procurement  documents  must  be  developed  such  that 
vendors  with  solutions  based  on  approved  products 
from  any  evaluation  scheme  could  compete.  Such 
procurement  documents  would  contain  phrases  such 
as:  the  proposed  solution  shall  comply  with  all 
requirements  for  ITSEC  F-Bl,  E3.  Systems 
integrator  teams  would  then  design  a  solution  which 
was  composed  of  subsystems  and  products  which  met 
the  requirement  in  the  most  cost  effective 
combination. 

Of  course,  the  solution  to  any  large 
requirement  usually  requires  combining  (evaluated) 
security  products  into  more  complex  systems.  The 
analysis  of  the  total  problem,  with  security 
considerations,  requires  that  the  security  team  be 
conversant  with  the  entire  architecture,  each  of  the 
evaluation  schemes,  and  the  products/technology 
evaluated  under  each  scheme. 

When  the  requirement  is  stated  in  terms  like: 
the  offerors’  solution  must  be  capable  of  evaluation 
at  the  TCSEC  C2  level,  the  integrator’s  problem  is 
compounded.  Products  which  have  been  evaluated 
according  to  one  scheme  may  not  be  acceptable,  or 
meet  the  analogous  level  in  the  others.  The  team 
must  consider  the  most  cost-effective  path  to 
satisfying  the  requirement  and  that  path  may  require 
including  an  evaluation  (or  re-evaluation)  of  a  product 
or  assurance  documents  in  the  overall  cost. 


What  Do  Multiple  Evaluations  Mean  to  Systems 
Integrators? 

Vendors  having  products  which  have  been 
evaluated  on  more  than  one  scheme  will,  of  course, 
be  reluctant  to  draw  distinctions  among  criteria. 
They  are  in  business  to  sell  products,  not  necessarily 
to  solve  a  given  specific  operational  problem. 

Micronyx,  an  international  vendor  of  a 
product,  TriSpan,  have  listings  in  the  UK  Certified 
Security  Products  List  (UKSP),  and  the  U.S. 
Evaluated  Products  List  (EPL).  The  product  was 
advertised  in  Infomatics  [4]  as  UK  Government 
Certified  and  U.S.  NCSC  Certified.  Indeed,  the  1 
October  1991  UKSP  lists  TriSpan  version  1.2130  as 
meeting  UKL2  (Independently  Tested).  The  U.S. 
NCSC  EPL-SUM-89/007,  however,  gave  the  product 
an  overall  rating  of  TCSEC  Class  D,  for 
Identification  and  Authentication  (I&A), 
Discretionary  Access  Control  (DAC),  and  Audit, 
stating  that  it  does  not  meet  the  assurance  and 
documentation  requirements  for  a  higher  rating. 

A  trusted  systems  integrator  contemplating 
using  such  a  product  to  meet  an  overall  requirement 
of  any  of  the  equivalent  evaluation  ratings,  as 
described  earlier,  must  be  able  to  accurately  estimate 
the  costs  to  develop  assurance  and  documentation 
which  may  be  required  by  the  accrediting  or 
approving  official. 

Causes  for  Ambiguity 

An  international  systems  integrator  is  faced 
with  ambiguity  caused  by  several  factors.  Each  of 
the  principals  involved  with  these  criteria  has 
autonomy  to  construct  evaluation  schemes,  and 
maintain  separate  lists  of  evaluated  products. 
Currently  Canada,  the  UK  and  the  U.S.  have  such 
schemes  in  place  and  soon  there  likely  will  be  six 
lists. 

Technology  Export  Restrictions 

Six  lists,  mostly  kept  by  the  Intelligence 
community,  implies  loads  of  red  tape  to  export 
technology,  even  if  the  techniques  are  already  in 
documented  use  in  the  intended  market.  A  striking 
example  of  this  now  is  the  Data  Encryption  Standard 
(DES)  controversy.  All  the  major  U.S.  vendors  of 
systems  like  International  Business  Machine 
Corporation  (IBM),  Digital  Equipment  Corporation 
(DEC),  Hewlett  Packard  (HP),  etc,  incorporate  DES 
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in  the  security  functions  of  their  offerings.  The  DES 
modules  must  be  removed,  or  very  costly  negotiations 
must  be  engaged  to  allow  selling  their  products 
outside  the  U.S.,  even  though  DES  is  well 
documented,  and  other  sources  for  it  exist  outside  the 
U.S.  Similar  restrictions  are  being  applied  to  any 
products  rated  B3  or  higher  under  the  TCSEC. 

Language  Translation 

Additionally,  the  six  lists  are,  of  course, 
maintained  in  the  native  language  of  the  keepers. 
The  nuances  described  earlier  in  this  paper  are 
exacerbated  with  translation.  Each  scheme  is  likely 
to  have  an  array  of  ’interpretations  and  guidelines* 
such  as  now  exist  in  the  U.S.  scheme.  Both  the 
CTCPEC  and  ITSEC  contain  requirements  which  are 
not  found  in  the  TCSEC,  and  will  therefore,  require 
additional  explanation  of  how  these  requirements 
could  be  met  by  U.S.  EPL  products. 

Standards  and  Security  Requirements 

The  inherent  ambiguity  which  must  be 
resolved  by  system  integrators  is  aggravated  by 
divergent  standards.  Security  products  are  built  to 
comply  with  criteria,  not  International  Standards 
Organization  (ISO)  standards.  When  both  standards 
and  evaluation  ratings  are  stated  as  requirements  in 
procurements,  the  system  integrator  must  usually 
make  a  compromise.  A  detailed  technical  knowledge 
of  both  will  be  required  to  develop  a  cost  effective 
solution  which  complies  with  the  intent  of  standards 
and  security  requirements. 

Generational  Problems 

The  multiple  evaluation  syndrome  is 
compounded  by  what  may  be  termed  the 
"generational”  problem.  As  criteria  and  schemes 
evolve,  and  products  are  evaluated,  systems 
integrators  must  know  when  the  evaluation  was 
completed  in  order  to  assess  its  usefulness.  The  UK 
evaluation  scheme  predates  the  ITSEC.  The 
CTCPEC  states  that  revisions  may  be  annual.  (In 
fact,  a  major  revision  to  the  CTCPEC  was  released 
between  the  time  that  this  paper  was  conceptualized 
and  written,  requiring  major  changes  to  this  paper 
which  superficially  describes  the  approach.  A 
wonderful  example  of  the  pitfalls  and  frustrations 
caused  by  generations  of  criteria.)  Additionally,  the 
U.S.  has  come  out  with  the  review  draft  of  a  new 
federated  criteria  that  replaces  the  TCSEC.  [12,  13] 


It  is  clear  that  procurement  professionals 
won’t  stay  abreast  of  these  changes.  Several  recent 
U.S.  procurements  required  solutions  to  comply  with 
the  1983  version  of  TCSEC,  and  it  is  very  difficult  to 
ascertain  which  products  were  evaluated  using  that 
criteria.  Product  vendors  will  naturally  strive  to  get 
commitments  from  evaluators  to  freeze  requirements 
before  evaluations  are  begun,  to  avoid  moving 
targets.  Subsequent  users  of  those  products  will 
necessarily  need  to  know  this  information  when 
developing  specifications  for  systems. 

Suggestions  for  Community 

This  paper  has  surfaced  a  number  of  issues 
which  result  in  the  following  suggestions  for  the 
community.  The  community  includes  product 
vendors,  evaluators,  national  certification  bodies, 
procurement  professionals,  accreditation  officials  and 
systems  integrators  —  to  name  a  few. 

"Keeper"  of  Evaluated  Products  Lists 

Find  a  ’keeper*  for  the  lists  of  nationally 
evaluated  products.  Everyone  can  profit  from  others’ 
work  if  some  registry  existed  which  points  to 
evaluations  conqjleted  internationally.  Some 
international  organization,  such  as  the  United 
Nations,  could  be  approached  to  maintain  this 
registry.  (It  appears  that  in  the  new  world  order, 
NATO  may  be  looking  for  some  jobs  to  do!) 

Reciprocal  Evaluations 

Develop  reciprocal  evaluations  using  the 
three  criteria.  If  a  registrar  could  be  found,  this  may 
open  the  way  for  the  negotiation  of  international 
mutual  recognition  of  evaluations.  There  is  some 
work  going  on  now  in  this  regard  between  the 
TCSEC  and  the  CTCPEC,  or  U.S.  and  Canada.  In 
the  near  term,  a  single  list  of  evaluated  products  is 
beyond  our  reach.  But  imagine  the  benefits  to  be 
gained  by  having  a  coordinated  worldwide  list  of 
evaluated  security  products! 

International  Standards  for  Trusted  Systems 
Criteria 

Develop  an  international  standard  for  trusted 
systems  criteria.  Even  better  than  a  single  list  of 
products  evaluated  against  several  criteria  would  be 
a  single  international  trusted  systems  criteria 
standard.  Is  anybody  trying  to  coordinate  the  various 
trusted  systems  criteria  and  the  ISO?  Clearly  the 
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community  will  profit  if  eventual  ISO  standards  in 
security  services  can  be  provided  by  evaluated 
products.  This  work  seems  to  be  beginning  and  is 
applauded. 

Universally  accepted  standards  are  vendor 
driven.  Vendors  naturally  push  the  standards  which 
they  do  offer  or  can  offer  in  their  own  products. 
But,  having  several  international  security  criteria,  the 
vendors  are  not  going  to  be  economically  able  to 
support  all  of  the  criteria.  It  is  therefore  important  to 
have  this  criteria  coordinated  closely  with  the 
international  standards  conunittees. 

Evolution  of  Technology 

Perform  more  analysis,  such  as  the  paper  in 
14th  National  Computer  Security  Conference  [1]. 
This  paper  compares  the  requirements  of  TCSEC 
Class  B3  and  ITSEC  example  class  F-B3,  ES.  The 
comparison  of  specific  products  to  potential 
evaluations  (Targets  of  Evaluation)  sheds  a  much 
needed  light  on  differences  and  similarities  in  these 
criteria.  The  real  burden  is  on  trusted  sv  ‘tns 
professionals  to  stay  abreast  of  this  evolutioi  in 
technology,  and  attempt  to  inform  the  rest  of  the 
community.  Such  participation  in  the  process  results 
in  more  consistent  practices  internationally  and 
diminished  ambiguity  in  reporting  results. 
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DEVELOPING  FUTURE  ACQUISITION  MANAGERS 
WITH  BEHAVIORAL  SIMULATIONS 
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ABSTRACT 

The  Defense  Systems  Management 
College  (DSMC)  is  continually  searching 
for  innovative  training  approaches  and 
methods  to  prepare  the  acquisition 
workforce  for  the  management  and 
leadership  challenges  of  the  future.  The 
traditional  classroom  lecture  has 
gradually  given  way  to  the  seminar,  case 
study  and  now  the  simulation.  This 
paper  will  highlight  a  new  simulation 
methodology:  behavioral,  in-basket, 

organizational  simulations.  A  variety  of 
these  simulations  are  now  being 
developed  and  used  in  all  aspects  of  the 
DSMC  curriculum.  The  paper  will  lay  out 
the  general  characteristics  of  simulation 
training  as  well  as  the  unique  aspects  of 
behavioral  simulations  as  they  are 
employed  at  DSMC.  The  variety  of 
DSMC  simulation  applications  will  be 
explored  along  with  the  contribution 
these  simulations  are  making  in 
addressing  acquisition  management 
issues  and  challenges  of  the  future. 

BACKGROUND 

For  many  years,  cold  war  politics,  the 
superpower  arms  race  and  resulting  big 
defense  budgets  fueled  a  massive  U.S. 
Defense  acquisition  program.  However, 
with  the  dissolution  of  the  communist 
block,  heightened  international  economic 
competition  and  the  advent  of  a  new 
Democratic  administration,  U.S.  Defense 
acquisition  programs  and  personnel  will 


likely  be  trimmed  to  their  lowest  levels  in 
recent  history.  In  this  smaller  acquisition 
workforce  of  the  future,  an  even  greater 
premium  will  be  paid  for  developing  top 
flight  acquisition  managers  who  can 
operate  effectively  in  a  highly 
constrained  resource  environment.  Just 
as  the  new  acquisition  environment  calls 
for  acquisition  managers  to  develop  new 
and  more  effective  management 
approaches,  so  too  will  the  organizations 
which  train  these  future  managers  need 
to  develop  new  and  more  effective 
training  methods. 

Recent  research  has  confirmed  what 
managers  and  many  educators  have 
believed  for  years;  adults  learn  best 
when  they  perceive  that  the  leaning  is  of 
value  to  their  life  and  their  work 
environments.  The  work  of  Malcolm 
Knowles  indicates  that  adult  learning 
primarily  is  self-motivated;  adults  learn 
what  they  need  to  know  when  they 
understand  it  to  be  important,  with  a 
strong  emphasis  on  application. 
Instructors/facilitators  working  with  adults 
cannot  directly  transfer  knowledge  or 
skills  to  the  adult  learner;  at  best  those 
charged  with  the  mission  of  educating 
adults  can  serve  as  guides,  as  managers 
of  the  learning  process,  in  creating  the 
environment  so  that  adults  can  engage 
in  self-directed  learning. 

This  suggests  that  for  such  an 
environment  to  be  optimal,  it  should 
involve  some  elements  that  simulate 
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real-life  experiences  as  closely  as 
possible,  and  integrate  this  "learning  by 
doing"  with  theoretical  hypotheses. 

It  is  within  this  context  that  the  Education 
Department  at  the  Defense  Systems 
Management  College  (DSMC),  Fort 
Belvoir,  Virginia  has  been  involved  over 
the  past  seven  years  in  using  behavioral 
simulations  in  teaching  and  research 
activities  focusing  on  developing 
managerial  and  executive  skills.  Student 
feedback  has  supported  our  premise  that 
such  simulations  are  powerful  learning 
tools  that  can  concentrate  over  a  short 
period  of  time  what  can  take  much 
longer  to  learn  whether  on  the  job  or  in 
a  more  "traditional"  classroom 
environment.  Benefits  of  using 
behavioral  simulation  training  are 
summarized  in  Figure  1 . 

The  first  simulation  used  by  DSMC  is  the 
Looking  Glass  management  simulation 
which  was  developed  by  the  Center  for 
Creative  Leadership.  Looking  Glass  is 
used  as  an  elective  in  the  20-week 
Program  Management  Course  (PMC) 
and  as  a  featured  part  of  the  3-week 
Executive  Management  Course  (EMC). 
The  commercial  manufacturing  scenario 
is  not  closely  related  to  that  found  in  a 
Defense  acquisition  organization. 
However,  the  simulation’s  managerial 
issues  have  considerable  parallel  to 
those  faced  by  Defense  acquisition 
managers.  The  commercial  scenario 
also  provides  a  "neutral  playing  field"  for 
Defense  managers  to  focus  on 
developing  their  personal  skills  (instead 
of  concentrating  on  content  issues  in  the 
simulation). 

Based  on  the  positive  response  of  both 
faculty  and  students  to  the  Looking 
Glass  simulation,  two  behavioral 


simulations  were  developed  in 
conjunction  with  the  Management 
Simulation  Projects  (MSP)  Group  at  New 
York  University.  The  first  simulation 
focuses  on  key  content  issues  involved 
in  contract  management,  and  is  set  in  a 
government  contracting  office.  This 
simulation  is  used  as  an  instructional  tool 
in  a  mandated  two-week  course  on 
systems  acquisition  for  contracting 
personnel  assigned  to  a  major  weapon 
system  program  office  or  who  spend  at 
lead  half  of  their  time  supporting  a  major 
weapon  system  program.  It  is  also 
available  as  an  elective  to  students 
taking  the  PMC  and  EMC  classes. 

The  second  simulation  is  designed  to 
deal  with  management  behaviors  in  the 
context  of  a  food  company  (Foodcorp  II). 
This  simulation  focuses  primarily  on 
obsen/ing  management  behaviors  using 
Baldridge  and  Excelsior  Quality 
Management  Award  Criteria  as  a 
baseline.  At  present,  plans  call  for 
offering  this  simulation  as  an  elective  in 
the  PMC,  and  incorporating  it  into  an 
executive  development  program  for 
acquisition  managers  being  designed  by 
DSMC’s  Education  Department. 

The  Education  Department  is  also 
developing  a  new  project  management 
simulation  consisting  of  a  series  of 
integrated  vignettes  which  will  require 
participants  to  demonstrate  key  program 
management  competencies.  A  NASA 
lunar  mining  base  was  chosen  as  the 
scenario  to  establish  an  acquisition 
related  situation  but  a  content  neutral 
(non-DoD)  backdrop  to  focus  on 
assessing  and  developing  participants' 
managerial  and  executive  skills. 


Figure  1 .  Benefits  of  Behavioral  Simulations 


0  Are  inherently  interesting  and  motivating 


o  Allow  tailored  assignments  to  fit  student  needs 


0  Allow  students  to  experience  the  different  stakeholder 
perspectives  that  affect  the  outcome  of  major 
organizational  decisions  and  actions 


0  Give  the  students  a  feeling  (or  gestalt)  of  what  It  is  like  to 
manage  In  a  complex,  realistic  management  environment. 


o  Put  the  focus  squarely  on  the  leadership  and  management 
skills  found  to  be  most  critical  to  organization  and  program 
success 


0  Provide  real-time  feedback  from  multiple  sources  (self, 
peers  and  facilitators  as  well  as  comparison  with  prior 
simulation  experiences) 


0  Encourage  transition  from  classroom  learning  to  job 
application 
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BEHAVIORAL  SIMULATIONS 

A  behavioral  simulation  is  an  intensive 
experience  emphasing  learning  by  doing, 
and  providing  time  for  personal  reflection 
and  feedback  from  both  the  simulation 
facilitator  and  other  participants. 
Frequently,  behavioral  simulations 
involve  a  variety  of  roles,  each  related  to 
the  key  issues  undergirding  the 
simulation.  Each  role  contains  extensive 
information  on  past  activities, 
correspondence  on  current  practices, 
decision  situations,  and  critical  issues 
faced  by  the  organization.  Through 
observing  such  behaviors  as  problem 
solving,  decision  making,  and  priority 
setting,  simulation  participants  are 
provided  feedback  on  their  behaviors, 
and  are  then  able  to  make  self- 
assessments  as  to  the  efficacy  of  such 
behaviors. 

Behavioral  simulations  are  vehicles  or 
technologies  that  will  allow  us  to  hone  in 
on  the  interpersonal  behaviors  and 
dynamics  among  people  as  strategic  and 
tactical  objectives  emerge-issues  which 
involve  coordination,  articulation,  and 
integration  among  individuals,  work 
teams  and  outside  stakeholders.  As 
DoD  continues  to  deal  with  the  realities 
of  shrinking  resources,  revised  objectives 
and  new  missions,  acquisition  managers 
are  forced  to  face  the  paradox  of 
balancing  daily  short  term  demands  with 
long  term  strategic  issues  and  an  ever 
changing  climate. 

Behavioral  simulations  provide  realistic 
and  focused  but  neutral  environments 
which  allow  participants  to  develop  plans 
to  meet  the  organizational  challenges, 
and  then  reflect  on  these  behaviors  as 
they  received  peer  and  facilitator 
feedback.  The  need  to  receive  and 


provide  data;  to  apply  written  and  oral 
communication  skills;  to  set  priorities; 
allocate  resources  and  specify  goals; 
and  manage  and  negotiate  often 
conflicting  goals  and  objectives  are  all 
inherent  to  a  well  designed  behavioral 
simulation. 

PROGRAM  DESIGN 

The  design  of  the  behavioral  simulation, 
to  be  effective,  must  emerge  from  the 
goals  and  objectives  of  the  experiential 
learning  method,  and  be  compatible  with 
the  goals  and  objectives  of  the 
educational  program  within  which  the 
simulation  is  nested.  At  best, 
simulations  are  tools,  perhaps  even 
enabling  technologies.  Their  value  is 
optimized  when  well  integrated  into  the 
goals,  objectives  and  conceptual 
underpinnings  of  the  educational 
programs  they  support. 

The  following  are  some  questions,  the 
answers  to  which  inform  and  stimulate 
design  considerations: 

0  What  are  the  desired 

educational  (behavioral) 
outcomes? 

0  How  much  time  is  to  be 

allocated  to  the  simulation? 

0  What  pre-reading  or  other 

preparation  is  required? 

0  What  special  logistical 

requirements  must  be 
considered? 

0  What  are  they  key  follow¬ 

up  and  assessment 
activities? 
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How  many  participants  can  BIBLIOGRAPHY 

be  accommodated? 


0  What  special  facilitating 

skills  are  needed? 

As  an  example.  DSMC's  current  Looking 
Glass  Design  is  illustrate  in  Figure  2. 

COMPARISONS 

Behavioral  simulations  emphasize 
learning  through  doing,  as  compared 
with  case  studies  which  focus  on  the 
application  of  cognitive  learning  or 
computer-based  simulations  which  focus 
on  analysis-based  learning.  Case 
studies  with  their  emphasis  on 
application  and  synthesis  in  the  cognitive 
domain  require  participants  to  function  in 
an  almost  consultant  like  role. 
Computer-based  simulations  cause 
participants  to  compete  within  the 
constraints  of  a  programmed  model; 
effectiveness  is  measured  against  the 
model's  design  rather  than  through  the 
interactions  and  behaviors  of 
participants.  The  similarities  and 
differences  between  these  learning 
vehicles  are  summarized  in  Figure  3. 

SUMMARY 

Currently,  Defense  acquisition 
organizations  are  competing  in  uncertain, 
challenging,  risk-filled  environments. 
Therefore,  there  Is  the  ever  constant 
need  to  balance  short  term  tactical 
objectives  against  longer-term  strategic 
issues  and  a  multitude  of  constraints. 
Behavioral  simulations  allow  future 
acquisition  managers  to  experience  and 
receive  feedback  on  how  these 
challenges  can  best  be  addressed. 
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LOOKING  CLASS  SIMULATION 
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CASES 


Figure  3.  Comparison  of  Case  Studies,  Computer  and  Behavioral  Simulations 


Case  Studies 

Computer  Simulations 

Behavioral  Simulations 

Materials 

One  case 

One  case  with  computer 
instructions 

Individually  tailored 
cases 

Viewpoint 

Students  view  case 
from  same  perspective 
(as  external  consultants) 

Students  view  case  from 
same  perspective  (as 
competitors) 

Each  student  has  a 
different  role,  information 
c.nd  perspective 

Approach 

Analyze  problems  and 
propose  solutions 

Students  compete  with 
each  other  and  against 
computer  model 

Students  assume  roles 
with  instructors  as 
observers/role-players 

Class  Session 

Instructor-led  seminar 

Interaction  with  computer 

Interactive  experience 

Results/Product 

Logical  analysis  and 
acceptable  solutions 

Computer  analysis  and 
comparison  with  other 
participants 

Personal  feedback  on 
both  content  (what  was 
done)  and  process  (how 
it  was  done) 

Evaluation 

External  assessment 

External  assessment 

Self  assessment 
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CUSTOMER  SATISFACTION  IN  DEFENSE  ACQUISITION 
Forrest  C.  Gale 

Defense  Systems  Management  College 


ABSTRACT 

This  paper  reports  Interim 
results  of  ongoing  action 
research  on  customer  satis¬ 
faction.  The  research  is 
designed  to  assess  the  results 
of  the  application  of 
behavioral  change  technology 
which  addresses  the  following 
questions:  (1)  What  are  the 
components  of  customer -supplier 
interaction?  (2)  What  are  the 
mechanisms  whereby  the 
negotiation  and  interaction 
between  customer  and  supplier 
result  in  the  customer-supplier 
agreement  complex?  (3)  What 
are  the  generic  structural, 
environmental,  experience- 
based,  and  species-specific 
precursors  to  customer-supplier 
agreement?  (4)  What  is  the 
nature  of  the  behavioral 
phenomena  we  call 
"satisfaction”?  And  (5)  what 
motivates  a  customer  (and — 
reciprocally — a  supplier)  to  be 
"satisfied"?  this  paper 
reports  data  and  information 
obtained  from  a  random  number- 
selected  sample  of  494  leaders, 
managers,  and  professionals  in 
military  service  organizations, 
civilian  government  agencies, 
and  in  private  sector  business 
organizations.  This  sample  was 
selected  out  of  a  base 
population  of  3872  individuals 
encountered  in  the  conduct  of 
custom  educational  experiences 
and  consulting  assignments 
during  the  1987-1993  time 
interval.  The  reported  interim 
results  provide  a  developing 
framework  within  which  one  can 
begin  to  perceive  the  outline 


of  a  practical  customer 
satisfaction  strategy  which  is 
based  on  the  formal  application 
of  behavioral  change  theory. 
This  is  seen  by  the  author  as 
an  integrating  contribution  to 
the  leadership  activity  of 
strategic  thinking  and 
planning,  wherein  a  major 
component  and  focus  of  that 
activity  becomes  the  formalized 
strategizing  of  behavioral 
change . 

INTRODUCTION 

The  fundamental  goal  of 
all  quality-related  activity  in 
organizations  is  customer 
satisfaction.  This  basic 
purposing  premise  underlies  the 
quality  theories  of  W.  Edwards 
Doming,  Joseph  Juran,  Kaoru 
Ishikawa,  and  many  others  cited 
referentially  in  the  literature 
of  Quality.  This  single 
purposing  objective,  achieved 
via  the  transformation  (by 
human  and  human-machine 
enactment  of  processes)  of 
resources  into  products  and 
services,  differentiates  the 
managerial  and  leadership  focus 
afforded  by  the  Quality 
paradigm  from  that  of  all  the 
other  traditional  enacting 
frameworks  of  leadership  and 
management.  As  an  example  of 
the  traditional  framing  of 
management  and  leadership 
objective,  consider  that  in 
many  organizations  the 
objectives  of  cost,  schedule, 
technical  product  performance, 
and  efficiency  are  the 
overriding  focus  of  management 
attention,  leadership  vision. 
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and  the  supporting  culture.  In 
many  such  organizations,  these 
outcomes  are  seen  as  ends  in 
and  of  themselves,  rather  than 
as  by-products  of 
organizational  (process 
enactment)  activity  focused  on 
customer  satisfaction  via  the 
delivery  of  product  and  service 
quality. 

Systems  theory  tells  us 
that  one  must  understand  the 
goals  and  objectives  of  a 
system  to  understand  the 
system.  If  the  system  is 
Quality,  can  it  be  anything 
less  than  critically  important 
to  understand  the  consummate 
goal  of  quality  effort,  which 
is  customer  satisfaction?  To 
date,  the  literature  of  quality 
has  focused  on  the  tools, 
techniques,  theories  and 
philosophies,  and  to  a  lesser 
extent  on  the  systems  of 
quality  enactment  and  the 
sustaining  environment.  Little 
of  substance  has  been  written 
on  the  enacting  mechanisms  of  a 
quality  system's  most  singular 
purpose. 

Without  an  understanding 
of  satisfaction  and  its 
attendant  dynamics,  the  quality 
practitioner  is  in  the  position 
of  applying  quality  theory, 
tools,  techniques,  and  systems 
in  an  environment  characterized 
by  a  critically  flawed  premise; 
that  one  must  only  satisfy 
customer  "needs”  in  order  to 
attain  the  super-ordinate 
system  goal.  The  results  of 
the  research  reported  in  this 
paper  provide  a  much  more 
powerful  and  empowering  set  of 
insights — grounded  in  a 
theoretical  framework — upon 
which  to  base  behavioral  change 
strategy  centered  on  customer 
satisfaction. 


The  Customer-Supplier  Agreement 

The  research  undertaken  in 
this  project  involved 
interviewing  and  interacting 
with  defense  acquisition 
professionals  and  a  variety  of 
support  staff.  Through  the 
hybrid  medium  of  the  structured 
interview  process  and  a  series 
of  small  team  process 
interventions  utilizing  a  focus 
question  protocol  and  image- 
based  communication  tools,  data 
was  collected  that  represents 
the  actual  practice  of 
conceiving,  planning, 
negotiating  and  enacting  a 
formal  customer-supplier 
agreement.  The  content  of 
these  agreements  varied  widely, 
but  the  process  elements 
invariably  cited  by  respondents 
as  providing  a  framework  for 
successful  enactment  of  the 
agreement  became  obvious  as  we 
statistically  combed  the  data 
base  citations.  As  a  set  of 
attributes  of  agreement,  these 
elements  clearly  delineate  what 
is  required  to  make  a 
successful  quality  agreement 
and  then  create  customer 
satisfaction  (and  supplier 
satisfaction)  in  the  enactment 
of  the  subject  agreement. 

There  follows  (Figure  1)  a 
listing  of  these  attributes  of 
agreement.  The  attribute 
citations  were  accepted  only 
from  those  respondents  who 
indicated  successful  enactment 
of  the  subsequent  agreement, 
and  who  also  indicated  that 
their  reciprocal  partner 
(customer  or  supplier) 
specifically  stated 
"satisfaction"  with  the 
enactment.  Further,  only  those 
respondents  who  indicated  their 
own  "satisfaction"  with  the 
outcome  of  the  enactment  were 
included  in  results  of  figure  1 
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and  in  the  citation  acceptance 
process.  This  dual 

satisfaction  population 
numbered  3872  individuals.  A 
random  sample  of  494  of  these 


individuals  who  then  selected 
for  more  extensive  analysis, 
the  results  of  which  are  cited 
throughout  this  paper. 


CUSTOMER- SUPPLIER  AGREEMENTS 


Process  Element 


Respondent  Citations  (%) 
Supplier  Custcroer  (^) 


Specific  ID  of  Product/ Service  98 

Specific  ID  of  Customer  as  "Customer**  74 

Customer  Classification  (Internal/External)  67 
Customer  Attributes  Model  Construction  56 

Customer  Quality  Model  Construction  61 

Specifically  Seek/ Determine  Reciprocal 

Wants  (Requirements)  85 

Specifically  Seek  Reciprocal  Needs 

(Desires)  77 

Determine  in  Totality  What  Will  Satisfy  64 

Determine  Current  Process  Capabilities  51 

Determine  Potential  Process  Capabilities  19 

Determine  Available  Resources  (To  Process)  82 

Inform  on  Available  Resources  (To  Process)  46 

Inform  on  Current  Process  Capabilities  36 

Assessment  of  Internal  Environmental 

Factors  77 

Assessment  of  External  Environmental 

Factors  80 

Jointly  Determine  Quality  Factors  44 

Negotiated  Quality  Agreement  82 

Formalized  (Explicit)  Quality  Agreement  91 

Renegotiation  Required  During  Enactment  64 

Agreed  to  Concurrent  Process  and  Product 

Design  70 

Informed  Customer  of  Process  During 

PDCA  Cycle  51 

Continually  Sought  Feedback  on  Environ¬ 
mentally  Induced  Quality  Factor  Changes 
During  PDCA  Cycle  38 

Delivered  Product/ Service  With  Agreed- 

Upon  Quality  Factors  90 

Sought  Continuous  Feedback  on  Satisfaction  85 

Continuous  Customer  Education  on  Product 

Capabilities/Improvements/Opportunities  79 

Continuous  Customer  Education  on  Process 

Capabilities/Improvements/Opportunities  46 

Continuous  Customer  Education  on  Resource 

Availabilities,  Adequacy,  Quality  Impacts  86 


91 


60 

55 

39 

27 

4 


33 

73 

34 
85 
88 
73 

53 


4 

81 

44 
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Respondent  citations  (%) 


Continuously  Sought  Product  Improvement 
Through : 

-  Internal  Environmental  Search 

-  External  Environmental  Search 

-  Ongoing  Customer  Feedback 

-  Opportunistic  Technology  Search 

-  Teaming  With  Customer 
Continuously  Sought  Process  Improvement 

Through; 

-  Internal  Environmental  Search 

-  External  Environmental  Search 

-  Ongoing  Customer  Feedback 

-  Opportunistic  Technology  Search 

-  Stakeholder  Teaming  With  Customer 
Specific  ID  of  Supplier  as  "Supplier" 
Supplier  Classification  (Internal/ External) 
Supplier  Attributes  Kodel  Construction 
Supplier  Quality  Model  Construction 
Articulate,  Quantify,  Record  Needs 
Articulate,  Quantify,  Record  Wants 
Inform  Supplier  of  Needs 

Inform  Supplier  of  Wants 
Identify/Articulate  NON  Product/ Service 
"Satisfaction"  Attributes 
Inform  Supplier  of  NON  Product/ Service 
"Satisfaction"  Attributes 
Inform  Supplier  of  Current  Product/ 

Service  Use  Environment: 

-  Influences 

-  Change  Rates 

-  Future/ Forecasts 

Seek/Internalize  Information  on  Supplier 
Process  Capabilities 

Seek/Internalize  Information  on  Supplier 
Resource  Availabilities 
Inform  Supplier  of  Product/ Service 
Boundaries  and  Constraints: 

-  Economic 

-  System-Derived 

-  Political/Social 

Continuous  Monitor  Supplier  Progress  (PDCA) 
Continuous  Feedback  to  Supplier  on: 

-  Environmentally  Induced  Quality 

Factor  Changes 

-  Needs /Wants-Induced  Quality 

Factor  Changes 


95 

25 

66 

95 

47 


59 

43 

68 

95 

26 

4 


82 

64 


22 


5 

82 

78 

40 


4 

26 

75 

41 

95 

63 

13 

11 

88 

79 

98 

92 

67 

43 


44 

4 

12 

49 

29 


81 

66 

51 

78 


53 

62 
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Respondent  Citations  (%) 


Process  Element  Supplier  Customer  r%^ 

Provided  Continuous  Feedback  to  Supplier  on: 

-  Quality  Performance  Data  —  64 

-  Introduction/ Installation  —  82 

-  Test  (Initial)  —  61 

-  Volume  Usage  —  43 

-  User  Experience  —  79 

-  Customer  Satisfaction  —  77 

-  Process  Improvement  Opportunities  —  7 

Seek/ Internalize  Education  on: 

-  Supplier  Product/Service  Capabilities  —  24 

-  Improvement  Opportunities  —  11 

-  Supplier  Process  Functioning  —  60 

-  Supplier  Process  Capabilities  —  39 

-  Supplier  Process  Improvement 

Opportunities  —  5 

-  Supplier  Resource  Availabilities, 

Adequacy,  Quality  Impacts  —  11 


FIGURE  1  -  ELEMENTS  OF  SUCCESSFUL  CUSTOMER-SUPPLIER  AGREEMENTS 


Though  the  data  is 
preliminary  and  the  research  is 
ongoing,  the  elements  for 
successful  customer-supplier 
agreement  are  manifest  in  the 
results  evident  to  date. 
Specific  identification  of  the 
transacted  product  or  service 
is  requisite.  Customer  and 
supplier  roles  in  each  portion 
of  the  agreement  must  be 
clearly  and  specifically 
outlined.  Customer 

classification  (internal/ 

external)  must  be  explicated. 
Suppliers  must  seek  out  and 
exactly  determine  customers' 
needs  and  wants;  it  appears  to 
be  slightly  less  important  for 
customers  to  seek  out  suppliers 
needs  and  wants  though — as  we 
shall  see  as  we  later  examine 
the  psychology  of  quality 
agreements — both  are  critically 
important  in  achieving 
••satisfaction'',  as  the 
customer -supplier  agreement  is 
a  boat  in  which  both  customer 
and  supplier  ride  the  tide  of 
mutual  satisfaction  to  a 


successful  conclusion.  There 
is  a  saying  in  quality  that  the 
data  appears  to  validate:  "It 
takes  a  world  class  customer  to 
make  a  world  class  supplier." 

Both  customer  and  supplier 
in  successful  quality 
agreements  appear  to 
articulate,  quantify,  and 
record  needs  and  wants. 
Suppliers  appear  to  be  much 
more  involved  in  determining 
the  totality  of  satisfaction 
(outside  the  realm  of  direct 
product  or  service) ,  and  are 
somewhat  more  active  in 
investigating  the  reciprocal 
needs  and  wants  of  the  opposite 
constituencies.  Customers  are 
uniformly  more  vigorous  in 
informing  suppliers  of  their 
needs  and  wants  than  are 
suppliers  similarly  informing 
customers.  The  latter 

inclination  did  not  seem  to 
harm  performance  and-- 
ultimately — satisfaction  on  the 
agreements  cited. 


213 


Suppliers  cited  the 
determination  of  process 
resources  as  a  dominant  success 
factor,  while  successful 
customers  seem  not  to  seek  out 
such  information.  Assessment 
of  external  environmental 
factors  was  a  success  element 
of  both  suppliers  and 
customers,  but  internal 
environmental  assessment  was 
frequent  only  in  successful 
supplier  constituencies. 
Nevertheless,  successful 
customers  frequently  cited 
providing  continuous  feedback 
to  suppliers  on:  (1) 
product/process  introduction, 
installation;  (2)  user 
experience  with  the  subject 
product /process;  and  (3)  on 
satisfaction  with  performance 
on  the  agreement. 

The  negotiation  of  a 
quality  agreement  rather  than 
dictation  of  the  quality 
factors  by  edict  or  fiat  was 
frequently  cited  by  both 
successful  customers  and 
suppliers.  Moreover,  an  even 
higher  percentage  of 
respondents  cited  the  practice 
of  formally  and  explicitly 
recording  the  agreed-upon 
quality  factors,  no  matter  how 
seemingly  trivial  the 
agreement.  Nearly  three- 
quarters  of  all  respondents 
also  cited  the  requirement  to 
renegotiate  these  factors 
during  the  course  of  enactment 
of  the  agreement.  This  was 
apparently  facilitated  by  an 
agreement  by  more  than  two- 
thirds  of  the  responding 
suppliers  to  design  processes 
and  p r odu c t s / s e r V i ce s 
concurrently  whenever  new 
products  or  services  were 
involved.  The  latter  practice 
would  appear  to  be  a  result  of 
heightened  customer  awareness 
of  the  concurrent  engineering 


process  and  its  applicability 
to  a  wide  variety  of  both 
product  and  service 
developments . 

The  communication 
interface  between  customer  and 
supplier  and  the  dynamics  of 
that  interface  received 
considerable  emphasis  in  the 
respondent's  citations. 
Successful  suppliers 
overwhelmingly  sought 
continuous  feedback  on 
satisfaction  from  customers, 
while  the  reverse  was  true  in 
less  than  50%  of  the  instances 
cited  by  customer  respondents. 
In  the  realm  of  product 
improvement,  successful 
suppliers  continuously  sought 
out  opportunities  via 
organization-internal 
environmental  search, 
opportunistic  technology 
search,  and — to  a  lesser 
extent--through  ongoing 
customer  feedback.  Customers 
sought  product  improvement 
through  external  environmental 
search  (what  is  out  there?  Who 
else  is  doing  something 
better?) ,  and  through  the 
mechanism  of  feeding  this 
information  back  to  responsive 
suppliers.  Successful 

respondent  customers  most 
frequently  cited  information 
flow  to  suppliers  in 
product/ service  boundaries  and 
constraints  in  economic,  and — 
to  a  lesser  extent — system- 
derived  areas  of  concern. 
Finally,  successful  customers 
provided  continuous  feedback  to 
suppliers  on  need  and  want- 
induced  quality  factor  changes. 
Coupled  with  the  better 
communication,  these  self  same 
successful  customers  frequently 
identified  and  articulated  non¬ 
product/service  “satisfaction” 
attributes  to  suppliers,  and 
included  these  factors  in  the 
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negotiated  quality  agreement. 

Successful  suppliers 
overwhelmingly  sought 
continuous  process  improvement 
through  opportunistic 
technology  search  and — to  a 
lesser  extent — through  ongoing 
customer  feedback.  Respondent 
customers,  however,  cited 
ongoing  feedback  to  suppliers 
as  their  primary  process 
improvement  mechanism.  To 
improve  customer  knowledge  and 
investment  in  their  products 
and  processes,  successful 
suppliers  pursued  continuous 
customer  education  on  product 
capabilities,  improvements,  and 
opportunities;  and  on  resource 
availabilities,  adequacy,  and 
quality  impacts.  Successful 
respondent  suppliers  did  not 
seek  to  educate  customers  on 
their  processes  in  spite  of 
what  appears  to  be  a  rather 
substantial  opportunity  to  do 
so.  Alternatively,  less  than 
half  of  successful  respondent 
customers  continuously  sought 
to  obtain,  internalize  and  act 
upon  supplier  process 
information.  Nevertheless, 
nearly  four  out  of  five 
respondent  customers  reported 
monitoring  supplier  progress 
during  the  enactment  of  the 
quality  agreement. 

An  interesting  anomaly 
appeared  in  the  respondent 
customer-supplier  data  with 
respect  to  the  perception  of 
satisfaction  resulting  from  the 
enactment  of  the  subject 
quality  agreement,  and  with 


respect  to  the  fidelity  of  that 
enactment.  Though  all 
customer-supplier  respondents 
cited  "satisfaction”  as  the 
outcome  of  the  agreement 
enactment,  only  90%  of  the 
respondent  suppliers  and  81%  of 
the  respondent  customers 
reported  delivering  product  or 
service  with  the  agreed-upon 
quality  factors.  This  anomaly 
induced  the  researchers  to 
probe  the  issue  of  satisfaction 
itself,  using  the  structured 
interview  and  small  group 
intervention  format  previously 
described,  together  with  the 
application  of  theories  of 
motivation  to  test  the 
hypothesis,  "The  motivation  to 
be  satisfied  can  be  explained 
by  Deming's  hypothesis  on 
quality  delights  and  by  Vrooms 
theory . " 

Components  of  Customer 
Satisfaction 

Slightly  more  than  80%  of 
the  customer-supplier 
respondents  in  our  sample 
reported  experiencing  more  than 
two  levels  of  satisfaction  from 
successful  enactment  of  their 
subject  quality  agreement.  In 
order  to  explore  this  finding, 
we  attempted  to  obtain  a 
generic  understanding  of  these 
levels  by  requesting  an 
experience-based  definition 
from  respondents.  The 
following  set  of  definitions 
(Figure  2)  describes  the 
composite  response: 
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Needs:  Elements  of  product  or  service  satisfaction  below 
which  basic  ability  (usefulness)  is  compromised  (i.e.,  not  useful). 
For  our  suppliers,  being  able  to  meet  needs  constituted,  by  their 
own  description,  the  entree  (poker  chip;  the  necessary  ante)  into 
the  competitive  market  place. 

Wants;  Elements  of  product  or  service  satisfaction  that 
constitute  the  majority  of  the  grist  of  the  competitive  market 
place.  These  were  described  by  our  respondents  as  "nice  to  haves; 
features;  more  for  the  same/ less  compensation  (both 
physical/economic  and  psychological) ;  but  always  above  the  level  of 
basic  utility." 

Delights  fafter  Demina^ :  Elements  of  product  or  service 
satisfaction  offered  by  an  "inside-the-customer-mind-and-heart" 
supplier;  creatively/ innovatively  generated  by  a  leading-edge 
supplier  from  superior  knowledge  of  both  self-generated  product  or 
service  (and  process)  and  implied/ inferred/ sensed  higher  order 
desires  not  even /vet  expressed  bv  the  customer.  This  was  further 
described  by  respondents  as:  "I  know  they  (customer)  would  love 
this  if  they  had  it;  create/make  the  market;  lead  direction,  pace, 
substance,  texture  of  market  through  the  ability  to 
opportunistically  capitalize  on  superior  customer  and 
product/ service  knowledge". 

FIGURE  2  -  LEVELS  OF  SATISFACTION  DEFINITIONS 


Thus,  it  can  be  said  that 
customer  satisfaction  equals 
needs  (market  entree) ,  plus 
wants  (market  competition) , 
plus  delights  (lead  or  create 
the  market)  .  It  was  no 
surprise  to  our  competition- 
weary  suppliers  nor  our  vaguely 
dissatisfied  customer 
respondents  to  realize  that 
supplying  needs  alone  rarely 
provides  complete  or  long  term 


customer  satisfaction.  In  this 
vein,  some  respondents  noted 
that  customers  ceaselessly 
search  for  a  higher  level  of 
satisfaction,  and  will  leave  in 
droves  when  a  better  mousetrap 
for  equivalent  compensation 
appears  on  the  horizon.  Sic 
transit  loyalty.  Such 
discussions  have  led  us  to 
propose  a  model  for  the  three 
apparent  levels  which  we  have 
come  to  call  "The  Golden 
Staircase. " 
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THE  CUSTOMER  SATISFACTION  STAIRCASE 


THE  MARKETPLACE  AND 
CUSTOMER  ACTIONS  OVER 
TIME  INEXORABLY  DRIVE 
DELIGHTS  TO  WANTS  AND 
WANTS-ULTIMATELY-TURN 
INTO  BASIC  NEEDS  FOR 
ANY  LONG  TERM  PRODUCT 
OR  SERVICE 


OUT 

OF 

BUSINESS 


SUPPLIERS  ARE  FORCED 
BY  COMPETITIVE 
EXIGENCIES  TO  STRIVE 
TO  LEAD  THE  MARKET. 
FOR  HIGH  TECH,  RAPIDLY 
CHANGING  PRODUCT  OR 
SERVICE  CATEGORIES 
WITH  EVER  SHORTENED 
PRODUCT  LIFE  CYCLES  OR 
"TIME-TO-MARKET" 
INTERVALS,  THIS 
STRIVING  BECOMES  A 
SURVIVAL  IMPERATIVE. 


FIGURE  3  -  THE  GOLDEN  STAIRCASE 


Examination  of  the  Golden 
Staircase  concept  reveals  that 
environmental  forces  in  the 
market  are  in  opposition  for 
customers  and  suppliers. 
Suppliers  continuously  seek  to 
drive  up  the  staircase  to  lead 
the  market,  and  thus  dominate 
it.  This  position  of 
satisfaction  provides  the 
greatest  rewards  and  -  as  we 


shall  soon  discuss  -  the 
greatest  potential  for  self 
validation  of  worth  of  the 
supplier  of  any  of  the  three 
levels.  Competition  is  "dog- 
eat-dog"  at  the  "need"  level, 
and  for  suppliers  there  is  a 
high  risk  of  de-validation,  for 
-  after  all  -  many  other 
suppliers  can  provide  what  the 
supplier  lodged  at  this  level 
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can  provide,  including  the 
needed  quality.  Price  and 
intangibles  often  determine 
one's  position  in  such  a 
capability  situation.  Survival 
is  not  assumed  for  competitors 
at  this  level,  where  going  out 
of  business  equates 
psychologically  to  death,  the 
ultimate  de-validation  of  self 
worth.  The  "wants”  level  of 
satisfaction  is  where  most 
market  competitors  are,  and  the 
risk  of  being  dislodged  from 
one's  niche  (providing  more 
"wants”  for  economic  or 
equivalent  psychic 
compensation)  is  continuous. 
But  at  least  one  can  search  for 
and  create  a  niche,  though  many 
of  the  terms  of  the  engagement 
are  dictated  by  market  forces 
and  peer  competition. 
"Creating  the  market”  (The 
"Delights”  Level)  is  the  place 
to  be.  All  world  class  quality 
suppliers  have  their  sights  set 
on  this  level.  Our  respondents 
in  this  research  have  told  us 
that  the  greatest 
"satisfaction”  as  a  supplier 
comes  from  mastering  customers 
and  ones  own  products/ services 
and  processes  to  the  extent 
that  one  can  define  the  future 
and  then  create  it.  Thus,  the 
drive  in  the  supplier  world  is 
ever  toward  the  provision  of 
"delights”  and  its  concomitant 
customer  ( and  supplier) 
satisfaction. 

Customers,  on  the  other 
hand,  continuously  seek  to 
drive  delights  down  the 
staircase  to  the  lowest 
("need”)  level.  Literally, 
today's  delight  is  tomorrow's 
want  is  next  week's  need. 
Everyone  must  have  what 
delights  others,  for  in  the 
delight  level  of  satisfaction 
is  the  greatest  validation  of 
self  worth,  the  invention  of 


product  or  service  for  self. 
Our  respondents  told  us  that  if 
a  peer  were  being  "delighted,” 
they  wanted  to  be  "delighted” 
also.  Such  innovations  quickly 
became  part  of  other  quality 
agreements  and  soon  became 
"wants”  for  many,  and — 
ultimately — needs.  Pity  the 
poor  competitor  at  the  need 
level  of  satisfaction.  The 
ante  is  continually  being 
raised. 

Thus,  even  though  the 
customer-supplier  respondents 
in  our  sample  were  having  their 
stated  needs  and — in  some 
cases — wants  met  through 
enactment  of  customer-supplier 
agreements,  and  were 
"satisfied,”  their  level  of 
satisfaction  varied.  Likewise, 
the  quality  agreements 
contained  both  needs  and  wants 
in  many  cases,  said  our 
respondents.  And  the  evolution 
of  many  agreements  included 
"delights”  not  envisioned  at 
the  time  of  the  original 
agreement  by  either  customer  or 
supplier.  Thus,  these  variants 
can  be  seen  to  provide  a 
plausible  explanation  for  the 
statistical  variance  of  the 
reported  resultii  with  respect 
to  both  full  delivery  of  the 
agreed-upon  quality  factors  and 
the  accompanying  satisfaction 
of  both  customer  and  supplier. 

Motivation  to  be  Satisfied 


Personal  precursors  to 
satisfaction  include: 
environmental 
( including 
conditioning 
establishment) ; 
group  dynamics 


influences 
cultural 
and  goal 
experience, 
(bionics) ; 


paradigms  (perceptual  frames 
and  selective  perception) ; 
needs;  values;  species  specific 
factors  (including  genetics) ; 
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conation;  physiological 
influences;  etc.  Figure  4 
provides  an  arrow  diagram 
representation  of  very  complex 
phenomena  that  are  outside  the 


boundaries  of  the  subject  of 
this  paper.  However,  the 
collective  influence  of  these 
framing  forces  cannot  be 
ignored . 


EXPERIENCES> 

PROTOMENTAL/STRUCTURAL- 
ARCHTYPES’ 

NEEDS 

VALUES - 3SkPERCEPTI0NS- 

PARADiCMS:::::===^\ 

BIONICS-"^  \ 


[-»>ENVIR0NMENTALS 


INTERNAL 

ENVIRONMENT 


CHOICES-^BEHAVK 


EXTERNAL 
ENVIRONMENT 


PERCEPTION  -  WHAT  IS  SEEN  (AS  ’SO’) 
CHOICES:  WHAT  IS  FOCUSED  ON 
WHAT  IS  DECIDED 
WHAT  IS  ACTED  UPON 


BEHAVIORS:  REACTIONS  MOMENT-TO-MOMENT 


FIGURE  4 


Central  to  an 
understanding  of  the 
satisfaction  dynamic  is  the 
concept  of  compensation. 
Compensation  in  its  essence  is 
much  more  than  economic;  it  is 
psychological,  and-- 
interestingly — psychometric. 
Applying  Vrooms  theory  to  the 
issue  of  compensation  in  a 
customer-supplier  agreement  and 


thence  to  the  motivation  to  be 
satisfied  may  provide  a 
framework  for  both  strategizing 
this  motivation  and 
quantitatively  determining  it's 
strength.  What  follows  is  a 
discourse  shared  with  our 
research  respondents  and 
experientially  validated  by 
them  as  they  framed  their 
success  (and  non-cited 
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failures)  with  customer- 
supplier  quality  agreements. 

Vroom  said  that  there  are 
three  components  to  the 
motivation  to  do  anything 
(i.e.,  to  do  is  to  make  a 
choice  or  change,  then 
behave/ act;  in  this  context, 
doing  nothing  equates  to 
"doing”  or  "behaving,"  moment- 
to-moment) .  These  three 


components  are:  (1) 
expectancy;  (2)  valence;  and 
(3)  instrumentality,  and  the 
theory  (Expectancy  Theory) 
states  that  one's  level  of 
motivation  is  a  product  of  the 
three  components. 

Definitions  of  these  three 
components  are  shown  in  Figure 
5,  below. 


EXPECTANCY  :  Examination  leading  to  the  self-perceived 
probability  P,(E)  that  a  potentiated  behavior/ act ion  will  lead  to 
ascertained  performance  (performance  can  be  equated  to  "success," 
as  defined  by  the  individual) . 

VALENCE  :  Examination  leading  to  the  self-perceived  degree 
of  alignment  of  a  potentiated  action/behavior  with:  (1)  one's 
needs;  and  (2)  one's  values  (behaved  priorities). 

INSTRUMENTALITY  ^1)  :  Examination  leading  to  the  self- 
perceived  probability  P,(I)  that  a  potentiated  behavior/ action  will 
have  one  or  more  ascertained  outcomes  (such  as  reward  or 
punishment),  both  valence-derived/psychological  (intrinsic)  and 
physical/external  (extrinsic) . 

FIGURE  5  -  VROOM 'S  THEORY 


Thus,  if  Motivation  =  M,  then  M 
=  (E)  (V)  (I)  and  M  =  P.(E)  x  V  x 
P.(I)  . 

Both  P,  (E)  and  P,(I)  can 
vary  between  0  and  1 ,  and 
Valence  (alignment)  between  +1 
(totally  aligned)  and  -1 
totally  opposed.  Classical 
need  theory  is  subsumed  in  the 
Valence  component.  Thus,  an 
act  totally  aligned  with  the 
dominant  need  in  one's  need 
complex  (Maslow)  can  be  said  to 


have  a  valence  of  +1,  while  the 
same  act,  opposing  one's 
behaved  value  system,  would 
have  a  valence  of  -1. 
Multiplying  those  component 
values  together  gives  us  a 
potential  valence  between  +1 
and  -1,  (-1  in  the  case  cited 
here) .  When  multiplied  by 
P,(E)  and  P,(I)  ,  the  product 
yield  a  psychometric  measure  of 
motivation,  and  hence  the 
motivation  to  be  satisfied. 
Figure  6  refers. 
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CUSTOMER  ACTION  MOTIVATION /SATISFACTION  PROFILI 
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NOTE:  VALENCE  IS  DETERMINED  FROM  A  PANOPLY  OF  WEIGHTED  VALUES, 
BOTH  POSITIVE  AND  NEGATIVE.  THE  VALUES  (WITH  WEIGHTS)  ARE 
DETERMINED  AND  ASSESSED,  GIVEN  A  SIGNED  VALUE,  NORMALIZED,  AND  THEN 
MULTIPLIED  BY  EXPECTANCY  (PERCEIVED)  AND  INSTRUMENTALITY 
(PERCEIVED)  PROBABILITIES  TO  YIELD  A  MOTIVATION  (AND  POTENTIAL 
SATISFACTION)  LEVEL  FOR  AN  OUTCOME  OR  OUTCOME  COMPLEjf. 
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The  Role  of  Self  Validation  in 
Satisfaction 

Underlying  each  of  Vroom's 
components  is  a  unifying 
concept  called  validation  of 
self  worth.  Our  respondents 
told  us  that  they  perceived 
themselves  as  "worth  more"  if 
they  perceived  that  the  quality 
agreement  they  have  negotiated 
is  "enactable"  (supplier)  and 
that  the  quality  factors  will 
accrue  to  the  recipient 
(customer) .  They  are  self 
validated  in  their  worth 
because  the  time,  energy,  and 
personal  investment  they  made 
paid  off;  i.e.,  was  not  wasted. 
They  are  more  likely  to  be 
satisfied  because  their  self 
perceived  probability  of 
success  (i.e.,  satisfaction)  is 
high. 

The  valence  of  successful 
customer-supplier  quality 
agreements  is  positive  because: 
(1)  the  behaved  values 
(priorities)  of  the  negotiants 
are  manifest  in  the  agreement 
and  its  behaved  enactment,  and 
the  needs  (psychological  as 
well  as  economic)  are  in  the 
negotiated  conditions. 
Fulfillment  of  such  agreements 
validates  the  negotiant's  self 
worth  in  the  same  manner  that 
doing  anything  aligned  with 
ones  needs  and  values  is  more 
self  validating  than  doing 
something  opposite  to  one's 
needs  and  values  (wherein  we 
are  de-validated  and 
demotivated) . 

The  instrumentality  of 
successful  customer-supplier 
quality  agreements  yields  a 
high  self  perceived  probability 
of  expected  outcome  precisely 
because  of  the  conditions 
inherent  in  the  process 
elements  most  often  cited  by 


our  research  sample.  It  is 
more  self  validating  to  achieve 
outcomes  as  expected  rather 
than  have  those  same  outcomes 
rendered  uncertain  by  a  lack  of 
the  very  process  elements 
addressed  in  the  successfully 
enacted  customer-supplier 
agreement.  Thus,  applying 
quality  theory  to  design  known 
performance  into  transformation 
processes  yields  not  only 
higher  instrumentality  but  also 
greater  validation  of  self 
worth;  we  are  less  validated  or 
even  de-validated  by 
deficiency -dominated  surprises. 

Summary 

We  have  thus  come  to 
tentatively  conclude  that 
behind  the  terms  of  customer- 
supplier  agreements  and  the 
precursor  behavioral  and 
physical  circumstances  of  their 
negotiation  and  enactment  is 
the  critical  factor  of  self 
worth,  and  the  opportunity  for 
self  validation  of  this  self 
same  worth  in  the  agreement. 
Our  research  has  led  us  to 
conclude  that  satisfaction  is  a 
self  validating  act.  As  such, 
its  conditions  can  be 
strategized.  The  behavioral 
change  that  is  the  transition 
from  dissatisfaction  to 
satisfaction  can  be  made  more 
probable  if:  (1)  customers, 
suppliers,  and  managers/ leaders 
will  establish  conditions  that 
increase  participant  self 
perceived  probability  that 
satisfaction  from  a  customer- 
supplier  agreement  can  be 
achieved;  (2)  customers, 
suppliers  and  managers/ leaders 
formally  assess  the  behaved 
priorities  (values)  and 
dominant  needs  (psychological) 
of  the  participants  (i.e.,  each 
other)  in  the  negotiation  and 
enactment  processes.  Moreover, 
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they  must  develop  strategies 
and  agreement  conditions  to 
insure  that  enactment  will  be 
in  consonance  with  the  values 
and  psychological  needs  of  the 
participants;  and  (3) 
customers,  suppliers,  and 
managers/ leaders,  by  virtue  of 
inclusion/application  of  the 
customer-supplier  agreement 
process  elements  discussed  in 
this  paper,  must  negotiate  and 
then  engage  to  ensure  that  the 
outcomes  of  the  enactment  are, 
indeed,  satisfying  and  self 
validating.  These  outcomes 
must  accrue  to  both  customer 
and  supplier  by  virtue  of  the 
high  (self  perceived) 
probability  of  their 
achievement.  Customer 

satisfaction  is  thus  seen  as  an 
outcome  of  not  only  the 
fulfillment  of  a  quality 
agreement  but  also  as  a 
consequence  of  a  carefully 
structured  and  orchestrated 
strategy  to  create  the 
conditions  leading  to  the 
highly  probable  consequence  of 
the  perception  of  the  increased 
self-worth  of  both  customer  and 
supplier;  all  this  within  the 
enveloping  framework  of  a 
customer-supplier  interaction 
whose  dimensions  are  only  now 
becoming  clear  to  us. 
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AND  IMPLEMENTING  AWARD  FEES  IN  SYSTEMS  ACQUISITION 
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ABSTRACT 

The  philosophy  of  providing  contractors  an 
award  fee  is  based  upon  the  premise  that  the 
potential  improvement  in  the  quality  of  con¬ 
tract  performance  offsets  any  additional  cost 
to  the  contract.  How  one  establishes  and 
allocates  fee  on  an  Award  Fee  contract  is 
critical  to  obtaining  the  best  performance  at 
the  most  significant  times.  Put  another  way, 
is  the  government  motivating  the  contractor 
at  the  right  time  for  the  most  efficient  cost? 
The  focus  of  this  article  is  to  describe  how 
proper  application  of  base  fee,  award  fee, 
and  rollover  concepts  can  be  implemented 
through  a  variety  of  approaches.  This  article 
examines  the  innovative  use  of  award  fees  in 
the  F-22  (Advanced  Tactical  Fighter)  Pro¬ 
gram  Cost-Plus-Award-Fee  (CPAF)  contract, 
and  in  the  Increased  Performance  Engine 
Program  Firm-Fixed-Price  with  an  Award 
Fee  (FFP/AF)  contract. 

INTRODUCTION 

Contracts  containing  award  fees  often 
result  in  better  communications  than  other 
types  of  contracts  between  the  government 
and  the  contractor  and  greater  contractor 
motivation  to  achieve  exceptional  contract 
performance.  These  attributes  are  nomially 
associated  with  the  process  of  monitoring 
and  evaluating  contractor  performance.  The 
creation  and  allocation  of  fees  on  award  fee 
contracts,  which  are  linked  to  the  perfor¬ 
mance  evaluation  process,  are  seldom 
viewed  as  critical  communication  and 
motivation  tools.  Rather,  many  organiza¬ 
tions  primarily  focus  on  the  subject/objective 
aspects  of  the  evaluation  process  and  fee 
determination. 


The  philosophy  of  this  article  is  to 
describe  how  proper  application  of  base  fee, 
award  fee,  and  rollover  concepts  can  be 
implemented  through  a  variety  of 
approaches.  This  article  examines  the  inno¬ 
vative  use  of  Award  Fee  in  the  F-22  Program 
Cost-Plus-Award-Fee  contracts,  and  in  the 
Increased  Performance  Engine  Firm-Fixed- 
Price  with  Award  Fee  (FFPIAF)  contracts. 

ESTABLISHING  THE  FEE  POOL 

A  Cost-Plus-Award-Fee  contract  is  a 
cost-reimbursement  type  of  contract  that  pro¬ 
vides  for  a  total  fee  consisting  of  (a)  a  base 
amount  (which  may  be  zero)  fixed  at  incep¬ 
tion  of  the  contract  and  (b)  an  award  amount, 
based  upon  a  judgemental  evaluation  by  the 
government,  sufficient  to  provide  motivation 
for  excellence  in  contract  performance  (FAR 
16.305).  Thus,  the  fee  pool  on  a  CPAF  con¬ 
tract  may  consist  of  both  a  ba.se  fee  and  an 
award  fee  or,  ju.st  an  award  fee.  When  estab¬ 
lishing  the  fee  pool  on  a  CPAF  contract,  the 
first  step  should  be  to  detemiine  the  max¬ 
imum  fee  pool  for  a  given  effort. 

in  order  to  calculate  the  maximum  fee 
pool,  one  must  obtain  an  estimated  cost  for 
all  of  the  work  required  within-the-scope  of 
the  contract.  To  ascertain  the  estimated  cost 
of  a  proposed  contract,  the  buying  activity 
may  rely  on  the  contractor’s  proposed  cost 
and/or  the  buying  activity’s  estimated  cost. 
In  the  Department  of  Defense  (DOD),  con¬ 
tract  cost-estimating  techniques  vary  greatly. 
The  DOD  uses  contract  cost-estimating  tech¬ 
niques  which  range  from  detailed  cost  ana¬ 
lyses  conducted  by  multi-functional  govern¬ 
ment  teams,  (i.e.,  Should  Cost  Reviews),  to 
contract  cost  estimates  based  upon  historical 
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costs  of  similar  items  and/or  "grass-roots" 
engineering  estimates.  The  type  of  cost 
estimating  technique  is,  or  should  be,  depen¬ 
dent  upon  the  size,  complexity,  and  other 
critical  aspects  of  each  contract  action. 
Whichever  technique  may  be  used,  it  should 
be  documented  as  thoroughly  as  possible. 

Once  an  estimated  contract  cost  is 
developed  for  a  planned  government  pro¬ 
curement,  under  a  cost-reimbursement  type 
of  contract,  it  should  be  multiplied  by  the 
applicable  statutory  limit  imposed  by  FAR 
15.903,  which  includes  the  following  fee 
limitations  for  the  respective  types  of  effort: 

(a)  Research  and  Development 
(R&D)  -  1 5%  of  estimated  cost, 

(b)  Production  and/or  Modifications 

—  10%  of  estimated  cost,  or 

(c)  Architect  and  Engineering  (A&E) 

—  6%  of  e.stimated  construction 
cost. 

The  result  of  the  above  calculations 
represent  the  maximum  (allowable)  fee  pool 
which  a  contractor  could  possibly  obtain  for 
a  given  government  contract. 

However,  there  is  no  rule  which 
requires  the  government  buying  activity  to 
establish  the  maximum  (allowable)  fee  pool 
as  -  "The  Fee  Pool"  —  for  the  proposed  con¬ 
tract.  In  fact,  all  buying  activities  should 
consider  numerous  other  factors  prior  to 
establishing  a  fee  pool,  including;  available 
funding;  complexity  of  the  effort;  proper 
incentive  (risk  vs.  Return-On- Investment 
{ROI)y,  and  the  amount  of  the  base  fee,  if 
any. 

Clearly,  there  is  no  single  approach 
required  by  the  FAR  for  establishing  a  fee 
pool  for  a  CPAF  contract.  Rather,  one  must 
logically  develop  an  approach  based  upon 
rational  business  sen.se.  We  suggest  the  fol¬ 
lowing  three-step  basic  approach,  di.scus.sed 
above,  and  outlined  below: 

(1)  Develop  a  "sound"  estimated  cost 
for  the  proposed  effort. 


(2)  Multiply  the  estimated  cost  times 
the  applicable  fee  limitation 
(FAR  15.903). 

(3)  Then,  consider  all  other  factors 
which  are  pertinent  to  fee  deter¬ 
mination  (i.e.,  available  funding, 
complexity  of  the  effort,  ROI, 
and  amount  of  base  fee). 

ESTABLISHING  THE  BASE  FEE 

As  stated  earlier,  a  base  fee  on  a  CPAF 
contract  is  a  fixed-fee  that  is  established  by 
the  buying  activity  upon  contract  award.  A 
base  fee  is  incrementally  paid  to  the  contrac¬ 
tor  regardless  of  its  performance  on  the  con¬ 
tract,  so  long  as  the  contract  is  not  ter¬ 
minated.  On  government  cost- 
reimbursement  type  contracts  the  actual  pay¬ 
ment  of  a  base  fee  typically  accompanies  a 
contractor’s  monthly  reimbursement  (by  the 
government)  for  "best  efforts"  of  actual  con¬ 
tractor  expenses.  Simply  knowing  what  a 
base  fee  is  and  how  it  is  paid  is  good,  but 
knowing  how  to  determine  a  base  fee 
amount  is  better! 

In  the  following  discussion,  we  have 
identified  three  common  concepts  to  deter¬ 
mine  an  appropriate  amount  of  base  fee 
(which  may  be  u.sed)  for  CPAF  contracts. 
Further,  we  have  identified  each  of  these 
concepts  for  ea.se  of  discussion.  These  con¬ 
cepts  have  been  used  by  various  government 
buying  activities  to  determine  an  appropriate 
amount  of  base  fee  for  CPAF  contracts. 

(1)  Marginal-Pcrformance-Level  (MPL) 
Concept 

Inherent  in  this  concept  is  the  idea  that 
base  fee  is  established  with  a  particular  qual¬ 
ity  or  level  of  performance  in  mind.  Base 
fees  are  established  taking  into  consideration 
the  various  profit  analysis  factors  covered  by 
FAR,  but  in  an  amount  commensurate  with 
that  level  or  quality  of  performance  categor¬ 
ized  as  minimum  acceptable.'  The  MPL 
concept,  to  determine  the  amount  of  base 
fee,  has  been  used  by  NASA  and  other 
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government  buying  activities  for  nearly  30 
years.  This  concept  contains  no  preset  limits 
to  the  amount  of  base  fee. 

(2)  Unallowable  Cost  Off-set  (UCO)  Con¬ 
cept 

The  DOD  FAR  Supplement  {DFARS) 
216.404-2  (c)(2)(B)  states,  "The  base  fee 
shall  not  exceed  three  percent  of  the 
estimated  cost  of  the  contract  exclusive  of 
the  fee."  The  DFARS  does  not  provide  the 
rationale  for  why  this  limit  is  required  on 
DOD  contracts;  however,  through  our 
research  we  have  concluded  that  several 
financial  aspects  caused  the  development  of 
this  UCO  concept.  One  of  the  most 
significant  financial  aspects,  considered  vital 
to  the  establishment  of  the  UCO  concept, 
was  and  is  the  business  strategy  of  all  con¬ 
tractors  to  offset  unallowable  costs  which 
they  incur  on  government  cost- 
reimbursement  types  of  contracts.2  Typi¬ 
cally,  Aerospace  contractors  hav'*  had  two  to 
three  percent  of  their  contract  costs  deemed 
unallowable  as  a  result  of  government  cost 
accounting  standards  and  audits.^  Thu.s, 
many  contractors  will  not  agree  to  enter  into 
a  cost-reimbursement  type  of  contract  unless 
they  are  assured  a  minimum  fee  of  two  to 
three  percent  of  the  estimated  cost.  The 
government  under  the  UCO  concept  would 
provide  contractors  a  minimum  base  fee 
guarantee  up  to  the  DFARS  limit  of  three 
percent  to  offset  contractor  unallowable 
costs,  thereby  preventing  a  contractor 
financial-loss  situation. 

A  base  fee  of  more  than  three  percent  is  pos¬ 
sible  but  requires  a  DFARS  deviation.  This 
base  fee  is  paid  to  a  contractor  even  if  the 
contractor’s  performance  is  considered  by 
the  government  to  be  submarginal. 

(3)  Zero-Base-Fee  (ZBF)  Concept 

During  the  pa.st  ten  years,  many 
government  buying  activities  have  esta¬ 
blished  a  ZBF  concept  or  policy.  The  ZBF 
concept  is  based  upon  the  assumption  that 
eliminating  a  base  fee  the  total  fee  pool. 


(consisting  solely  of  an  award  fee),  would 
provide  a  greater  incentive  for  contractors  to 
achieve  superior  performance.  A  ZBF  pol¬ 
icy  was  used  by  the  Air  Force  Systems  Com¬ 
mand  (AFSC)  buying  activities  for  many 
years 

When  examining  these  concepts  to 
determine  the  amount  of  base  fee,  it  is  obvi¬ 
ous  that  the  MPL  concept  provides  the  con¬ 
tractor  with  the  largest  possible  base  fee, 
while  the  ZBF  concept  provides  none. 
Therefore,  the  UCO  concept  is  often  con¬ 
sidered  to  be  a  fair  and  reasonable  comprom¬ 
ise  to  determine  a  base  fee  amount.  How¬ 
ever,  the  UCO  approach,  as  modified  by  the 
DFARS  215.974,  has  the  following  three 
constraints  which  appear  quite  clear,  yet  are 
often  misused  and/or  misunderstood. 

(1)  DFARS  215.974(b)  states: 

In  developing  a  fee  objective  for 
CPAF  contracts  the  contracting 
officer  shall  not  use  the  weighted 
guidelines  method  or  alternate 
structured  approach. 

This  requirement  seems  very  definitive,  yet 
despite  its  clear  wording  some  major  defense 
buying  activities  have  for  years  provided 
CPAF  policy  guidance  to  their  buying  offices 
to  use  an  alternate  structured  approach  to  the 
DD  Form  1547  (Weighted  Guidelines 
Method)  in  order  to  determine  the  amount  of 
fee  on  CPAF  contracts  (without  obtaining  a 
DFARS  deviation).^ 

(2)  DFARS  215.974(c)  states: 

Apply  the  offset  policy  in 
215.973(b)(2)  for  Facilities  Capi¬ 
tal  Cost  of  Money  (FCCM)  -  DD 
Form  1861,  i.e.,  reduce  the  base 
fee  by  the  lesser  of  one  percent  of 
total  costs  or  the  amount  of  facili¬ 
ties  capital  cost  of  money. 

The  rationale  for  reducing  the  base  fee  in 
this  manner  is  provided  in  DFARS 
215.973(2)(ii): 
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This  adjustment  is  needed  for  the 
following  reason:  The  values  of 
the  profit  factors  used  in  the 
weighted  guidelines  method  were 
adjusted  to  recognize  the  shift  in 
facilities  capital  cost  of  money 
from  an  element  of  profit  to  an 
element  of  contract  cost  (FAR 
31.205-10)  and  reductions  were 
made  directly  to  the  profit  factors 
for  performance  risk. 

Many  people  in  the  contracting  community 
interpret  this  paragraph  to  mean  that  there  is 
a  relationship  between  award  fee  and  DOD 
profit  policy.  This  is  incorrect.  The  com¬ 
monality  between  these  two  DFARS  is  in  the 
treatment  of  FCCM,  not  DOD  profit.  This 
interpretation  is  consistent  with  the  guidance 
contained  in  DFARS  215.973(2)  and 
215.974(c)  and,  is  also  consistent  with  the 
first  constraint  discussed. 

There  is  still  one  more  question  con¬ 
cerning  the  application  of  the  offset  policy 
for  FCCM.  Does  one  reduce  the  base  fee  by 
the  lesser  of  one  percent  of  total  costs  or  the 
amount  of  FCCM  —  before  or  after  one 
applies  the  three  percent  limit  required  by 
DFARS  216.404-2(c)(B)?  Based  on  our 
research  with  various  DOD  representatives, 
we  have  determined  that  there  are  no  official 
documents  (i.e.,  regulation,  policy,  etc.)  that 
deal  with  this  question.  It  is,  therefore,  .sub¬ 
ject  to  interpretation.  It  is  our  view  that  the 
interpretation  is  not  as  important  as  insuring 
that  one  does  not  violate  any  fee  limits 
(15%-R&D,  l()%-Production,  6%-A&E,  or 
3%-DOD  base  fee  limit).  In  other  words, 
where  one  ends  up  (i.e.,  not  exceeding  the 
limits)  is  more  important  than  how  one  gets 
there  (e.g.,  reducing  the  ba.se  fee  —  before  or 
after  -  one  applies  the  three  percent  DFARS 
limit)  as  long  as  one  indicates  how  FCCM 
was  considered  in  the  process. 

(3)  DFARS  215.974(d)  states: 

In  developing  a  fee  objective  for 
Cost-Plus-Award-Fee  contracts. 


the  contracting  officer  shall  not 
complete  a  DD  Form  1547. 

This  is  the  final  statement  in  DFARS 
215.974  and  clearly  communicates  that  the 
use  of  DOD  profit  policy  is  not  appropriate 
in  developing  a  base  or  award  fee  objective 
for  CPAF  contracts. 

These  three  additional  constraints 
imposed  by  DFARS  to  the  UCO  approach  to 
determining  a  base  fee  have  increasingly 
raised  questions  from  both  contractors  and 
government  personnel  as  to  their  exact  intent 
and  applications  for  CPAF  contracts.  As  dis¬ 
cussed  above,  there  is  no  single,  best 
approach  to  determining  a  base  fee  for  a 
CPAF  contract.  In  fact,  determining  a  base 
fee  for  a  CPAF  contract  is  largely  dependent 
upon  the  government  buying  activity’s 
specific  requirements  and/or  policies.  The 
real  challenge  to  CPAF  contracting  is  in  the 
creation  of  the  Award  Fee  -  How  much  is 
enough? 

CREATING  AN  AWARD  FEE 

For  years  major  defense  buying  activi¬ 
ties  have  provided  guidance  to  their  buying 
offices  to  u.se  objective  methods  in  order  to 
arrive  at  either  the  size  of  the  award  fee  pool 
or  the  amount  of  the  award  fee  determination 
on  CPAF  contracts.  In  many  instances,  this 
was  manifested  in  the  use  of  weighted  guide¬ 
lines  or  an  alternate  structured  approach  for 
these  detemiinations.  DFARS  215.974 
clearly  states: 

In  developing  a  fee  objective  for 
CPAF  contracts  the  contracting 
officer  shall  not  use  the  weighted 
guidelines  method  or  alternate 
structured  approach. 

This  paragraph  further  states  that; 

contracting  officers  shall  not  com¬ 
plete  a  DD  Form  1547  (Weighted 
Guidelines  Method). 

The  use  of  weighted  guidelines,  or  an 
alternate  structured  approach  to  arrive  at 
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either  the  size  of  the  award  fee  pool  or  the 
amount  of  an  award  fee  determination  on  a 
CPAF  contract,  is  inappropriate  because  it  is 
a  misapplication  of  the  DOD  profit  policy* 
Equally  important,  the  use  of  totally  objec¬ 
tive  methods  to  arrive  at  the  award  fee  pool 
or  awaid  fee  determination  is  a  contradiction 
of  the  concepts  that  underlie  and  support  the 
use  of  CPAF  contract.  Flexibility  is  needed 
to  arrive  at  both  an  award  fee  pool  and  award 
fee  determination  that  suit  the  circumstances 
of  a  particular  procurement.’ 

While  there  is  a  need  for  flexibility  in 
the  determination  of  an  award  fee,  there  is 
also  a  need  for  consistency  in  the  process. 
Consistency  and  flexibility  are  not  exclusive 
of  each  other.  Award  fee  ratings  must  be 
clearly  related  to  the  available  award  fee 
pool.  Consistency  does  not  mean  using  an 
unalterably  structured  approach.  Rather,  it 
means  clearly  defining  the  subjective  /  objec¬ 
tive  elements  of  the  flexible  approach  one  is 
using.  Defining  your  philo.sophy  in  the 
determination  of  award  fee  is  beneficial  to 
both  the  contractor  and  the  government. 

The  description  of  how  one  will  create 
and  administer  an  award  fee  is  the  essence  of 
communications.  While  there  should  be 
flexibility  in  the  application  of  the  philoso¬ 
phy,  the  philosophy  itself  should  be  con¬ 
sistently  applied  throughout  a  buying 
activity.  The  consistent  application  of  an 
award  fee  determination  will  improve  com¬ 
munications  and  have  an  impact  on  the 
entire  contracting  cycle  from  the  request  for 
proposal  to  the  completion  of  the  contract 
effort. 

The  authors  will  di.scu.ss  a  philo.sophy 
of  consistency  in  award  fee  ratings  that  can 
improve  the  communications  between  the 
government  and  the  contractor  and  better 
motivate  excellence  in  performance.  The 
point  of  this  discussion  is  not  to  relate  that 
any  one  philosophy  represents  a  preferred 
approach,  but  that  a  buying  activity  should 
have  a  clearly  defined  philosophy  and 


broadly  communicate  that  philosophy  to 
their  personnel  and  potential  sources. 

DEVELOPING  STANDARDS  FOR 
EACH  CRITERIA 

The  first  step  in  achieving  a  philosophy 
of  consistency  in  establishing  award  fees  is 
to  develop  performance  standards  for  each 
evaluation  criterion.  These  performance 
standards  should  be  written  so  they  describe 
what  is  required  to  meet  the  terms  and  condi¬ 
tions  of  the  contract.  Contractor  perfor¬ 
mance  normally  should  not  be  rewarded  at  or 
below  this  standard.  Performance  standards 
mu.st  relate  to  an  adjectival  rating/grade. 
Which  adjectival  rating  (e.g.,  satisfactory, 
marginal,  good)  one  uses  is  not  as  important 
as  how  clearly  it  is  described. 

This  concept  is  consistent  with  DFARS 
216.404-2  which  states  that: 

Normally,  award  fee  is  not  earned 
when  the  fee-determining  official 
has  determined  that  contractor 
performance  has  been  submarginal 
or  unsatisfactory. _ 

CONSISTENCY  IN  CREATING  AWARD  FEE  ; 

i 

! 

ESTIMATED  COST:  $100  MILLION 

MAXIMUM  FEE  POOL  (RiDh  16%  OR  16.000,000  [: 

BUYINO  ACTIVITY  AVQ.  FOR  CPFF:  7.6%  OR  7.600.000  1, 

EXAMPLE  FI:  I 

BASE  FEE;  3%  $3,000,000 

AVa  FIXED  FEE  MAX.  AWARD  FEE  \ 

AVWRD  FEE  RANGE:  0  TO  $4,600,000  TO  $12,000,000  } 

(37.6%)  (100%)  jj 

EXAMPLE  F2:  ' 

BASE  FEE:  0%  j; 

AVa  FIXED  FEE  MAX.  AmRD  FEE  {I 
AWARD  FEE  RANGE:  0  TO  $7,600,000  TO  $16,000,000 

(60%)  (100%)  !< 

1 

Figure  one 

There  should  be  one  adjectival  rating 
level  that  relates  to  the  description  in  the 
DFARS  when  an  award  fee  is  not  normally 
given.  In  other  words,  meeting  the  terms 
and  conditions  of  the  contract  should  not 
entitle  the  contractor  to  an  award  fee. 
Exceeding  the  terms  and  conditions  of  the 
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contract  is  excellence  and  should  be 
rewarded.  Proper  use  of  developing  stan¬ 
dards  for  criteria  bring  some  objectivity  into 
a  subjective  process.  Most  organizations 
that  use  CPAF  contracts  accomplish  this  step 
to  some  degree.  Thus,  the  process  of 
developing  clear  and  consistent  levels  of 
evaluation  standards  for  contractor  perfor¬ 
mance  improves  the  award  fee  process. 

RELATING  THE  STANDARDS  TO  AN 
AMOUNT  OF  AWARD  FEE 

The  next  step  is  to  relate  the  perfor¬ 
mance  described  in  the  adjectival  rating  for 
standards  to  an  amount  of  fee.  Doing  this 
does  not  reduce  the  flexibility  of  award  fee 
determinations.  It  simply  clarities  an 
organization’s  philo.sophy  for  creating  and 
determining  the  amount  of  award  fee.  Using 
your  buying  activity’s  average  Cost-Plus- 
Fixed-Fee  (CPFF)  contract  communicates 
one’s  award  fee  philosophy.  Of  course,  in  a 
CPFF  contract  the  government  assumes  the 
risk,  namely,  the  award  fee.  Nevertheless,  it 
is  still  an  excellent  tool  for  comparison.  Fig¬ 
ure  One  illustrates  how  a  particular  philoso¬ 
phy  of  award  fee  determination  can  be 
related  to  a  buying  activity’s  average  CPFF 
percentage. 

In  Figure  One,  the  buying  activity  has 
determined  that  if  the  contractor  meets  the 
terms  and  conditions  of  the  contract  in  a 
CPAF  contract,  it  should  earn  essentially  the 
same  fee  as  they  would  in  a  CPFF  contract. 
The  estimated  cost  is  $1(K)  million  with  a 
maximum  fee  pool  of  15  percent  (R&D),  or 
$15  million.  The  buying  activity’s  average 
CPFF  contract  is  7.5  percent  of  the  $1(K)  mil¬ 
lion  estimated  cost,  or  $7.5  million.  Exam¬ 
ple  number  one  in  this  Figure  has  a  base  fee 
of  three  percent  of  the  estimated  cost,  or  $3 
million.  If  the  contractor  achieves  a  rating 
that  equates  to  meeting  the  terms  and  condi¬ 
tions  of  the  contract,  it  will  earn  approxi¬ 
mately  the  same  as  it  would  under  a  CPFF 
contract.  This  would  amount  to  $4.5  million 


or  37.5  percent  of  the  available  award  fee 
pool  of  $12  million. 

In  example  number  two,  there  is  no 
base  fee  so  the  award  fee  pool  for  meeting 
the  terms  and  conditions  of  the  contract 
would  be  $7.5  million  or  50  percent  of  the 
award  fee  pool  of  $15  million.  The  approach 
one  would  use  with  award  fee  only  does  not 
have  to  be  the  same,  nor  are  the  authors 
implying  that  one  should  structure  an  award 
fee  philosophy  that  pays  a  contractor  the 
same  amount  for  meeting  the  terms  and  con¬ 
ditions  of  a  CPFF  contract  as  a  CPAF  con¬ 
tract.  The  focus  of  these  examples  is  not  to 
espou.se  a  philosophy  of  award  fee  determi¬ 
nation  but  to  illustrate  the  importance  of 
clearly  communicating  one’s  award  fee  phi¬ 
lo.sophy  and,  how  that  communication  will 
motivate  performance  excellence. 

This  step  in  creating  an  award  fee  is 
not  done  well  in  most  buying  activities. 
Contractors  generally  try  to  determine  each 
organization ’.s  philo.sophy  of  award  fee 
determination  by  intelligence  and  past 
experience.  This  is  usually  satisfactory  for 
contractors  that  know  the  particular  buying 
activity,  as  long  as  that  activity’s  philosophy 
does  not  change  and  is  consistent  with  its 
past.  It  is  not  beneficial  to  either  the  govern¬ 
ment  or  the  contractor  to  project  possible 
award  fee  detenninations  solely  on  supposi¬ 
tion  when  an  award  fee  determination  philo¬ 
sophy  can  and  should  be  clearly  communi¬ 
cated  in  such  a  way  that  it  will  motivate  con¬ 
tractors  to  superior  perfonnance. 

FEE  ALLOCATION  AND  ROLLOVER 

Fee  allocation  and  rollover  are  key 
motivators  for  excellence.  How  fees  are 
structured  on  CPAF  contracts  will  determine 
the  potential  level  of  motivation.  The  length 
of  the  evaluation  period  and  the  amount  of 
award  fee  available  for  each  period  can  vary. 
This  is  a  government  decision,  but  it  is 
proper  to  .seek  the  contractor’s  view.s.  Typi¬ 
cally  the  government  establishes  evaluation 
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periods  from  four  to  six  months  in  duration.* 
This  takes  into  consideration  the  contractor’s 
desire  to  shorten  evaluation  periods  so  they 
can  recover  their  award  fee  more  quickly  and 
the  government’s  desire  to  lengthen  the 
periods,  to  reduce  the  administrative  burden. 
Evaluation  periods  longer  than  one  yea»'  gen¬ 
erally  do  not  achieve  the  desired  benefits  for 
either  the  government  or  the  contractor. 

The  amount  of  award  fee  available  can 
vary  with  each  evaluation  period,  with  larger 
portions  made  available  on  evaluation 
periods  in  which  performance  criteria  are  of 
greater  significance.  If  a  contract  contains 
tests,  demonstrations,  or  milestones,  then 
allocate  a  certain  amount  to  the  event,  not  to 
the  evaluation  period.’ 

Figure  Two  is  an  example  of  how  one 
could  allocate  an  award  fee  pool.  In  this 
example,  the  authors  took  a  contract  with  a 
$100  million  estimated  cost  and  multiplied 
the  cost  times  the  statutory  limit  (percentage) 
for  an  R&D  contract  (15  percent)  to  achieve 
a  fee  pool  of  $15  million.  The  award  fee 
pool  was  then  determined  by  subtracting  the 
base  fee  of  $3  million  from  the  maximum 
fee  pool  of  $15  million  for  an  award  fee  pool 
of  $12  million.  This  example  contains  five 
award  fee  periods  of  equal  length  with  vari¬ 
ous  amounts  of  award  fee  available  based  on 
more  dollars  being  allocated  to  evaluation 
periods  in  which  criteria  were  of  greater 
significance.  Also,  there  was  an  additional 
$900,000  available  to  whatever  period  final 
acceptance  occurred. 

Unearned  award  fee  may  be  carried 
forward  for  possible  award  in  subsequent 
evaluation  periods.  However,  this  process, 
called  "rollover,"  "roll-forward,"  or  "recoup¬ 
ment,"  should  be  the  exception  and  not  the 
rule.  Rollover  has  certain  disadvantages 
regardless  of  how  or  when  it  is  used.  The 
disadvantages  are;  (a)  it  devalues  previous 
periods,  and  (b)  it  allows  contractors  to 
lower  performance  in  some  periods  without 
being  penalized.  In  certain  limited  situations 


the  advantages  to  the  government  of  using 
rollover  can  outweigh  the  disadvantages. 
Potential  advantages  can  be:  (a)  it  would 
increase  the  available  award  fee  pool;  (b)  it 
would  not  penalize  the  contractor  for  a 
learning-curve  situation;  and  (c)  it  could 
motivate  the  contractor  to  correct  problems 
from  previous  periods.’"  These  potential 
advantages  relate  principally  to  deliverable 
hardware  items. _ 

EXAMPLE:  (COST-PLUS-AWARD-FEE  CONCEPT) 

ESTIMATED  COST  MAXIMUM  EES*  FEES 

StOO  MILLION  X  IS  •  $15,000,000 

BA&L1£S% 

SlOO  MILLION  X  ^  .  $3,000,000 

AVAILABLE  AMARD  FEE  S12.000.0C0 


0  i 

20% 

1 

16% 

1  1 

10% 

a  *0 

10%  1  30% 

S2.400.000! 

S1,S00.000  { 

11.200.000 

1  S1.200.000  If 3.600.000' 

,  ’ 

i  » 

1  3 

i  . 

•  ADDITIONAL  SSOO.OOO  TO  WHATEVER  PERIOD  DEVELOPMENT  TESTING 
OCCURS. 

ADDITIONAL  tftOO.OOO  TO  WHATEVER  PERIOD  FINAL  ACCEPTANCE 
TESTING  OCCURS  (PHESUMAeLY  THE  LAST). 

2  " 

Figure  two 

Rollover,  therefore,  might  be  appropri¬ 
ate  in  an  Engineering  and  Manufacturirg 
Development  (E&MD)  contract  where  initial 
Operational  Test  and  Evaluation  {lOT&E) 
was  required  prior  to  transitioning  into  a 
lower-rate,  initial  production  on  another  con¬ 
tract.  It  would  be  appropriate  to  rollover  an 
award  fee  from  the  period  prior  to  the  period 
in  which  lOT&E  occurs  in  order  to  place 
greater  incentive  on  the  period  in  which 
lOT&E  occurs.  The  FAR  and  the 
government’s  supplemental  regulations  per¬ 
mit  the  use  of  numerous  methods  for  allocat¬ 
ing  fees  and  the  use  of  rollovers,  when 
appropriate,  on  CPAF  contracts  if  doing  so 
will  truly  assist  buying  activities  in  motivat¬ 
ing  contractors  to  achieve  exceptional  per- 
fomiance. 
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Implementing  Innovative  Approaches  to 
Award  Fees 

The  following  portion  of  the 
manuscript  shall  examine  two  distinctly  dif¬ 
ferent  acquisitions  by  discussing  their 
respective  innovative  approaches  to  award 
fees,  and  their  respective  lessons  learned. 
The  specific  programs  we  shall  review  are 
the  F-22  Engineering  and  Manufacturing 
Development  contract  and  the  Increased  Per¬ 
formance  Engine  (/P£)  production  contracts. 
All  of  these  contracts  were  awarded  by  Air 
Force  Materiel  Command  (AFMC), 
Aeronautical  Systems  Center  (ASC),  and  are 
contractually  administered  by  their  cognizant 
Defense  Plant  Representative  Offices 
(DPROs). 

Increased  Performance  Engine  (IPE) 
Award  Fee  Description 

Two  IPE  production  contracts  valued 
at  $  9(X)  million  each,  were  negotiated  as 
Firm-Fixed  Price  with  Award  Fee  (FFPIAF) 
type  contracts.  One  IPE  contract  is  between 
AFMC/ASC  and  General  Electric  (6'£),  and 
the  other  is  between  AFMC/ASC  and  Pratt- 
Whitney.  The  IPE  contacts  each  contain  a 
maximum  fee  pool  of  $2  million  per  year. 
The  award  fee  is  included  to  further  motivate 
each  contractor  to  exceptional  perfomiance 
in  order  to  improve  .safety,  quality,  reliabil¬ 
ity,  and  maintainability  of  controls  and 
accessories  for  their  IPE.  The  IPE  FFP/AF 
contracts  each  have  evaluation  periods  which 
are  six-months  in  duration.  The  IPE  con¬ 
tracts  are  scheduled  to  last  from  I  December 
1989  thru  30  November  1993.  The  .semi¬ 
annual  evaluation  periods  are  1  December- 
31  May  and  1  June-.30  November  of  each 
year. 

The  Fee  Detennining  Official  (FDO) 
for  the  IPE  contracts  originally  was  the 
Commander,  Aeronautical  Systems  Center 
iASdCC),  but  was  later  delegated  to  the 
Subsystem’s  System  Program  Office  {SPO) 
Systems  Manager,  ASC/SM.  The  award  fee 


evaluation  and  fee  recommendation  to  the 
FDO  are  made  by  an  Award  Review  Board 
{ARB)  composed  of  SPO  and  DPRO 
representatives  from  the  following  functional 
areas:  engineering,  manufacturing,  contract¬ 
ing,  logistics,  program  control,  quality,  and 
others.  With  these  basic  facts  concerning  the 
IPE  contracts,  let  us  discuss  four  of  their 
innovative  approaches  to  award  fees. 

IPE  Innovative  Approaches  to  Award 
Fees  ”  Award  Fee  Use  on  FFP  Contracts 

Seldom  has  an  Award  Fee  been  used 
on  a  multi-million  dollar  subsystem  contract 
in  conjunction  with  a  Finn-Fixed-Price  type 
of  contract.  In  addition,  when  an  award  fee 
is  combined  with  a  type  of  contract  other 
than  CPAF,  the  profit  or  fee  already  part  of 
the  contract  serves  as  the  equivalent  to  the 
ba.se  fee.  Thus,  the  IPE  contracts  do  not  con¬ 
tain  a  typical  base  fee.  as  commonly  u.sed  on 
CPAF  type  of  contracts. 

Few  Performance  Monitors 

The  Subsystem’s  SPO  propulsion 
office  managing  the  government  administra¬ 
tion  of  the  IPE  contracts  selected  only  a  few 
perfomiance  monitors  in  each  functional 
area  to  provide  input  to  the  government’s 
Award  Review  Board  {ARB).  Typically, 
contracts  of  this  size  and  complexity  have 
many  more  people  involved  in  evaluating  the 
contractor’s  performance.  The  Subsystem 
SPO  found  that  sometimes  fewer  govern¬ 
ment  performance  monitors,  who  are  well 
educated  and  trained  in  their  functional  area 
and  their  specific  perfomiance  monitoring 
responsibilities,  are  helier\ 

Highly  Empowered  ARB 

The  I-DO  for  the  IPE  contracts  lias 
highly  empowered  the  members  of  the 
Award  Review  Board  to  evaluate,  tailor  and 
summarize  perfomiance  monitors’  findings. 
Even  with  the  best  perfomiance  monitors,  it 
is  possible,  (even  likely)  to  have  a  variety  of 
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opinions  when  it  comes  to  subjectively 
evaluating  a  contractor’s  performance,  no 
matter  how  much  one  objectively  structures 
the  evaluation  process.  The  FDO  needs  an 
as  clear  and  concise  as  possible  recommen¬ 
dation  from  the  ARB,  concerning  the 
contractoi’s  performance  versus  the  Award 
Fee  Plan.  Highly  empowering  the  ARB 
allows  the  ARB  members  to  segregate  the 
chaff  from  the  wheat  and  provides  the  ARB 
with  the  flexibility  to  examine  the  whole 
effort  from  a  system  perspective,  rather  than 
merely  reporting  functional  inputs. 

Comprehensive  Contractor  Documenta¬ 
tion  of  Their  Performance 

The  Subsystem’s  SPO  propulsion 
office  decided,  rather  than  having  the 
government  team  of  evaluators  compile  data 
from  numerous  functionally  generated  data 
items  (Contract  Data  Requirements  List  - 
DD  Form  1423),  that  it  would  be  far  more 
efficient  and  effective  to  require  the  contrac¬ 
tor  to  prepare  a  comprehensive  perfomiance 
information  package  to  submit  to  the  govern¬ 
ment  for  evaluation.  Requiring  the  contrac¬ 
tors  to  submit  a  comprehensive  information 
package  documenting  their  actual  perfor¬ 
mance,  versus  the  award  fee  plan,  accom¬ 
plishes  many  objectives,  including: 
encouraging  multi-disciplinary  solutions, 
improving  management  indicators,  helping 
focus  the  contractor  on  key  issues,  helping 
ensure  access  to  accurate  information,  and 
creating  a  nearly  comprehensive  (excludes 
government  evaluation)  document  to  support 
evaluations,  should  an  outside  agency  ever 
audit  the  IPE’s  award  fee  efforts. 

IPE  Award  Fee  --  Le.ssons  Learned 

Despite  the  use  of  several  innovative 
approaches  and  numerous  time-tested 
approaches  to  award  fees,  the  IPE  contracts 
initially  encountered  a  few  all-too-common 
problems.  Remember,  "We  must  learn  from 
our  past  mistakes,  otherwise  we  shall  be 


doomed  to  repeat  them."  The  AFMC/ASC 
Subsystem’s  SPO  propulsion  office  offers 
the  following  five  lessons  learned: 

( 1 )  Update  the  Plan  (for  example) 

Do  not  be  caught  with  50%  of  the 
award  fee  allocated  in  the  Award  Fee  Plan  to 
field  performance  when  you  have  not  fielded 
the  equipment. 

(2)  Do  Not  Be  Afraid  to  Vary  the  Fee 

Do  not  be  trapped  by  your  previous 
contractor’s  performance  evaluations.  Give 
the  contractor  the  amount  of  award  fee  that 
the  contractor  deserves,  whether  they  like  it 
or  not! 

(3)  Develop  Effective  Evaluation  Standards 

Award  Fee  evaluations  should  not  vary 
dramatically  based  solely  upon  evaluator 
personalities.  Perfomiance  Standards  must 
be  established  and  clearly  communicated  to 
the  contractor  and  to  all  government  perfor¬ 
mance  monitors. 

(4)  Award  Fees  Must  Have  A  Message 

The  FDO  must  use  the  award  fee  pro¬ 
cess  to  clearly  communicate  to  a  contractor 
his/her  view  of  their  performance  and  where 
improvement  is  required. 

(5)  Round-off  Evaluator  Scores  and  he 
Prepared  to  Defend  Them 

Typically,  the  ARB  summarizes  per¬ 
formance  monitor  inputs  but  does  not  always 
round-off  the  scores  (i.e.,  93.5  points,  vs. 
85-95  points.).  Rounding-off  evaluator’s 
scores  to  either  point  ranges  or  to  an  adjec¬ 
tival  rating  (i.e..  Excellent,  Good,  Satisfac¬ 
tory,  etc.)  can  ease  the  evaluation  process  by 
eliminating  non-value-added  discussions 
over  point  disagreements.  If/when  evaluator 
.scores  differ  by  more  than  20%,  you  had 
better  reconcile,  if  possible,  the  differences; 
otherwi.se  you  will  have  to  defend  the  vali¬ 
dity  of  the  differences  to  the  FDO. 
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F-22  Award  Fee  Description 

The  F-22  program  has  two  Cost-Plus 
Award  Fee  contracts.  One  contract  valued  at 
$1.4  billion  is  between  AFMC/ASC  and 
Pratt- Whitney  for  the  engines  and  the  other 
contract  valued  at  $9.7  billion  is  between 
AFMC/ASC  and  Lockheed  for  the  weapon 
system.  These  contracts  contain  a  fee  pool 
of  approximately  $  1  billion.  The  award  fee 
is  included  to  motivate  each  contractor  to 
exceptional  performance.  The  criteria 
selected  by  the  SPO  are  overall  progress 
toward  integrated  weapon/engine  system 
development,  overall  schedule  performance, 
overall  cost  control,  and  other  program  con¬ 
siderations.  The  SPO  believes  that  these  v  *- 
teria  encompass  the  entire  effort  and  ensures 
that  the  contractor  is  motivated  to  work  the 
complete  system  rather  than  to  focus  on 
specific  areas.  The  F-22  award  fee  evalua¬ 
tion  periods  are  six-months  in  duration  (for  a 
total  of  17  evaluation  periods  in  engineering 
and  manufacturing  development  (EMD)) 
which  is  scheduled  to  run  through  1999.  The 
semi-annual  evaluation  periods  are  1 
October-31  March  and  1  April-3()  September 
of  each  year.  The  Fee  Determining  Official 
(FDO)  responsibility  has  been  delegated  to 
the  F-22  Program  Director.  Now,  let  us  dis¬ 
cuss  four  of  their  innovative  approaches  to 
award  fee. 

F-22  Innovative  Approaches  to  Award 
Fees  —  Award  Fee  Philosophy 

The  F-22  System  Program  Office 
(SPO)  uses  an  integrated  management  struc¬ 
ture  and  this  structure  is  reflected  in  their 
philosophy  of  award  fee.  Essentially,  the 
contractor  plans/commits  to  an  executable 
program  and  then  manages  to  his  plan.  The 
government  evaluates  and  distributes  award 
fee  according  to  how  the  contractor  has  per¬ 
formed  to  his  plan.  This  approach  is  unique 
in  two  aspects.  First,  it  gives  the  contractor 
the  flexibility  to  plan  the  program,  then 
manage  to  his  plan  versus  the  government’s 


plan.  Second,  full  compliance  to  the  plan 
results  in  an  award  of  100%  of  the  fee  avail¬ 
able  that  period.  Typically,  award  fee  is  not 
paid  for  just  meeting  the  terms  and  condi¬ 
tions  of  the  contract,  but  for  exceeding  those 
requirements.  However,  exceeding  require¬ 
ments  in  a  cost-plus  contract  can  result  in 
additional  cost  to  the  government.  To  ensure 
no  "gold  plating"  occurs,  the  F-22  SPO  set 
up  the  award  fee  so  that  100%  award  would 
be  granted  for  total  compliance  to  the  con¬ 
tract.  The  philosophy  of  the  SPO  is  that 
when  the  contractor  plans  /  commits  to  an 
executable  program  and  achieves  it,  then 
true  excellence  has  been  met. 

The  F-22  SPO  uses  the  adjectival  rat¬ 
ings  of  excellent,  very  good,  good,  satisfac¬ 
tory,  and  unsatisfactory.  This  is  not  unusual, 
but  the  fact  that  they  have  only  one  adjec¬ 
tival  definition  for  all  criteria  is  innovative. 
For  example,  the  definition  of  the  adjectival 
rating  of  excellent  is  -  high  probability  that 
a  quality  product  will  be  delivered  on  time 
and  on  budget.  This  definition  applies  to  all 
award  fee  criteria.  Usually,  a  unique 
definition  is  written  for  each  adjectival  rating 
for  each  criteria  in  an  award  fee  plan. 

Base  Fee  Exceeds  DFARS  Limitation 

The  DFARS  states  that  base  fee  shall 
not  exceed  three  percent  of  the  estimated 
cost  of  the  contract,  exclusive  of  fee.  The 
F-22  received  approval  from  the  Director  of 
Defense  Procurement,  Office  of  the  Under 
Secretary  of  Defense  (Acquisition), 
USD(A)DP,  for  a  deviation  to  allow  a  base 
fee  of  four  percent.  The  F-22’s  award  fee  is 
9%  for  a  possible  13%  fee  to  be  earned  by 
the  contractor. 

Extensive  Performance  Monitoring 

The  F-22  SPO  approach  to  monitoring 
contract  performance  is  that  award  fee 
as.sessments  are  inseparable  from  the  day- 
to-day  management  of  the  program.  Con¬ 
sistent  with  their  integrated  management 
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structure  approach,  each  Integrated  Product 
Team  (JPT)  and  each  functional  office  has  a 
performance  monitor  focal  point  (PMFP). 
Within  these  organizations  every  member  is 
a  performance  monitor.  In  addition  to  the 
SPO  this  includes  members  of  the  Defense 
Plant  Representative  Offices  (DPROs),  Air 
Combat  Command  (ACC),  Air  Training 
Command  (ATC),  and  others  that  are  part  of 
the  program.  Every  member  provides  writ¬ 
ten  input  to  the  appropriate  PMFP  on 
specific  areas  of  knowledge.  The  ATP  SPO 
philosophy  is  that  all  government  personnel 
involved  in  the  F-22  program  have  a  voice  in 
award  fee  evaluations.  This  high  visibility 
results  in  no  surprises  at  award  fee  assess¬ 
ment  time  if  communications  are  open  and 
frequent.  The  involvement  of  the  monitors 
in  this  approach  is  much  broader  than  most 
award  fee  plans  and  it  is  far  more  detailed 
and  structured. 

Expanded  Award  Fee  Documentation 

One  PMFP  from  each  IPT  and  each 
functional  office  collects  and  consolidates 
input  from  respective  areas  of  the  program 
for  use  during  the  evaluation.  The  PMFP 
prepares  and  coordinates  monthly  reports 
through  the  IPT/Functional  Director.  Addi¬ 
tionally,  the  PMFP  prepares  and  coordinates 
a  midterm  briefing  at  the  three  month  time 
frame  of  the  award  fee  period  through  the 
IPT/Functional  Director.  The  F-22  award 
fee  branch  maintains  all  official  award  fee 
files  and  database  and  oversees  the  adminis¬ 
tration  of  award  fee  procedures.  To  insure 
that  communications  are  occurring,  feedback 
to  the  contractors  is  conducted  monthly. 
This  approach  to  award  fee  documentation 
and  feedback  is  much  more  comprehensive 
than  is  usually  found.  Typically,  there  are 
informal  communications  going  on 
throughout  the  award  fee  period  at  the  moni¬ 
tor  level  with  very  little  structure.  Many 
government  organizations  also  have  a 
requirement  for  some  midterm  infomial 
evaluation  of  the  contractor.  The  depth  and 


quality  of  this  evaluation  varies  greatly  from 
organization  to  organization  and  contract  to 
contract. 

F-22  -  Lessons  Learned 

The  F-22  award  fee  contracts  are  off  to 
an  early  and  successful  beginning,  but  there 
are  always  areas  for  improvement.  The 
AFMC/ASC  F-22  SPO  offers  the  following 
lessons  learned: 

(1)  Do  Not  Be  Afraid  to  Revise  Criteria 

The  F-22  SPO  determined  that  their 
criteria  needed  to  reflect  a  product  emphasis 
so  they  changed  the  criteria.  They  were  not 
hesitant  to  change  the  criteria  so  that  it  better 
met  their  needs  and  their  desired  emphasis. 
Do  not  be  afraid  to  change  any  aspect  of  the 
award  fee  plan  if  it  better  motivates  the  per¬ 
formance  you  desire. 

(2)  Utilize  Defense  Contract  Management 
Command  (DCMC)  Support 

The  F-22  SPO  realized  the  value  of 
DPRO  involvement  and  made  a  point  of 
reinforcing  this.  The  new  DCMC  is  organ¬ 
ized  to  support  their  customers,  government  / 
military  buying  offices.  The  organizational 
structure  of  DCMC  is  composed  of  three 
directorates;  Contract  Administration,  Qual¬ 
ity  Assurance,  and  Program  and  Technical 
Support.  Through  the  FAR  delegation  pro¬ 
cess,  DCMC  can  serve  as  a  valuable  asset  for 
evaluating  contractor  performance. 

(3)  Write  It  Down!  Write  It  Down!  Write 
It  Down! 

Sooner  or  later  your  memory  will  fail 
you.  It  is  imperative  to  keep  detailed  written 
records.  Be  specific  in  your  examples  to 
include  the  impact  of  what  you  are  docu¬ 
menting.  Put  your  written  records  where  you 
can  find  them. 

(4)  Communicate!  Communicate!  Com¬ 
municate! 

Frequent  and  specific  <'ommunication 
with  the  contractor  and  other  government 
organizations  is  a  must  at  all  levels.  There 
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should  be  no  surprises  when  there  is  open 
communication.  Award  fee  should  be  a  part 
of  daily  management  of  the  program. 

SUMMARY  AND  RECOMMENDA¬ 
TIONS 

This  article  has  examined  the  following 
aspects  of  award  fee  contracting;  how  to 
develop  a  maximum  fee  pool;  various  con¬ 
cepts  for  establishing  a  base  fee;  different 
philosophies  and  actual  approaches  for  creat¬ 
ing  an  award  fee;  the  importance  of  develop¬ 
ing  and  relating  performance  standards  to  an 
amount  of  award  fee;  and  some  of  the  con¬ 
cepts  and  techniques  involved  in  fee  alloca¬ 
tion  and  rollover.  These  issues  do  not 
comprise  an  all-encompassing  list  of  items 
concerning  contracting  with  an  award  fee. 
Rather,  this  article  has  focused  on  some  of 
the  less  frequently  discussed  issues  of  con¬ 
tracts  with  award  fees  —  creating,  allocating 
and  implementing  award  fees. 

When  involved  in  award  fee  contract¬ 
ing  the  authors  offer  the  following  recom¬ 
mendations: 

Use  UCO  Concept  for  Base  Fee  (For 
CPAF  Only) 

This  concept  for  establishing  a  base  fee 
ensures  a  contractor  that  it  will  be  provided  a 
minimum  fee  guarantee  to  "offset"  contrac¬ 
tor  unallowable  costs,  thereby  preventing  a 
contractor  financial  loss  situation. 

Consistent  Buying  Activity  Award  Fee 
Philosophy 

Every  government  buying  activity 
should  have  a  clearly  defined  philosophy  and 
broadly  communicate  that  philo.sophy  to 
their  personnel  and  potential  sources.  How¬ 
ever,  remember  consistency  and  flexibility 
are  not  mutually  exclusive.  A  consistent 
buying  activity  philosophy  can  and  should 
provide  flexibility  to  arrive  at  an  appropriate 
award  fee  pool  and  award  fee  determinations 
that  suit  the  circumstances  of  each  contract 


action. 

Fee  Creation  as  a  Motivational  Tool  for 
Excellence 

There  is  no  single  approach  required 
by  the  FAR  or  DFARS  for  establishing  a  fee 
pool  for  a  contract  with  an  award  fee.  What¬ 
ever  approach  is  used,  remember  that  the 
purpose  for  creating  the  fee  pool  is  to 
motivate  the  contractor  to  excellent  perfor¬ 
mance.  To  this  end  we  suggest  the  following 
approach: 

(1)  Develop  a  "sound"  estimated  cost 
for  the  proposed  effort. 

(2)  Multiply  the  estimated  cost  times 
the  applicable  fee  limitation 
(FAR  15.903),  for  cost-type  con¬ 
tracts. 

(3)  Then  consider  all  other  factors 
which  are  pertinent  to  fee  deter¬ 
mination  (i.e.,  available  funding, 
complexity  of  effort,  ROI,  and 
amount  of  base  fee). 

Rewarding  Excellence 

What  constitutes  excellence  must  be 
clearly  defined.  When  excellence  is 
achieved,  it  should  be  appropriately 
rewarded. 

CONCLUSIONS 

The  use  of  award  fees  on  government 
contracts  are  complex  and  often  controver¬ 
sial.  They  require  a  significant  amount  of 
contractor  performance  evaluation  by  the 
government.  But,  simply  stated,  any  con¬ 
tract  is  a  communications  vehicle  to  express 
an  agreement  between  its  parties.  Contracts 
containing  award  fees  provide  contractors 
with  numerous  avenues  to  communicate 
openly  with  the  government  during  contract 
perfomiance.  Moreover,  award  fee  contracts 
provide  the  government  with  greater  input  or 
leverage  to  motivate  contractors  to  achieve 
exceptional  performance. 
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CHANGING  THE  SYSTEM 
WHAT  ARE  THE  IMPLICATIONS  OF  AN 
ACQUISITION  DEPARTMENT? 

ABSTRACT 

For  the  past  several  vears  there  has 
been  a  vocal  call  for  a  revolution  in 
the  way  the  DOD  acquires  its  major 
weapon  systems.  The  need  for  change 
within  the  DoD  acquisition  system  has 
been  articulated  by  a  diverse  set  of 
individuals  who  run  the  gamut  across 
the  ideological  political  spectniTii. 
While  the  entrenched  bureaucracy 
fights  to  delay  the  inevitable,  there 
can  be  little  doubt  that  theie  will 
be  significant  changes  once  the 
Clinton  Administration's  Defense 
hierarchy  faces  the  reality  of 
reduced  budgets.  a  shrinking 
workforce  and  a  new  strategy  for 
weapons  acquisition  which  no  longer 
relies  upon  the  production  and 
deployment  of  systems,  but  rather  the 
development  and  prototyping  alone. 
The  new  reality  will  foice  the  system 
to  change.  Som.e  of  this  change  is 
already  evident  in  the  restructuring 
of  the  individual  services.  Air 
Force  Material  Command  is  a  precursor 
of  change  that  is  inevitable.  De¬ 
fense  contractors  are  consolidating 
and  eliminating  those  divisions  that 
have  little  or  no  value  added  to 
corporate  needs. 

If  we  accept  the  inev itabi 1  it v  (and 
necessity)  of  change,  what  will  that 
change  be?  Streamlining  is  a  kev. 
If  L'.S.  long  terr:,  national  security 
needs  are  to  be  met.  innovative  and 
creative  planning  will  be  required. 
What  are  some  of  the  recommendations 
for  this  systemic  change,  and  what 
are  their  implications?  One  of  the 
more  innovative  proposals  advocates 
the  creation  of  an  Acquisition  Ser¬ 
vice,  similar  to  the  other  services 
(Army,  Navv  .S  Air  Force  •.  This  ser¬ 
vice  would  be  a  hybrid  of  the  mili¬ 
tary  and  civilian  services,  with  many 
of  the  more  promising  features  of 
each.  If  the  DOD  is  truly  committed 
to  the  implementation  of  the  princi¬ 


pals  of  Total  Quality  Managen.ent, 
then  It  seems  reasonable  that  there 
must  be  major  changes  in  the  acquisi¬ 
tion  process.  The  military  must  be 
removed  from  all  aspects  of  the  ac¬ 
quisition  process  except  from  those 
nonnallv  reserved  for  the  customer. 

This  paper  will  discuss  the  basic 
concept  of  the  Acquisition  Service. 
It  will  provide  a  rational  explana¬ 
tion  as  to  the  costs  and  benefits  of 
such  a  significant  chaiigi’  along  with 
the  statutorv  and  regulatory  revi¬ 
sions  required.  The  centralii'at  ion 
of  the  procurement  function  will  not 
eliminate  the  abuses  that  have  tran¬ 
spired  in  the  past,  however,  there 
will  be  an  inevitable  streamlining  of 
the  process  which  can  not  help  but 
realize  substantial  savings  to  the 
governnenl . 

The  Clinton  Administration  is  conniit- 
ted  to  the  revitalization  of  "Th>.’ 
views  expressed  in  this  article  are 
those  of  the  author  and  do  not  le- 
flect  the  official  policy  or  position 
of  the  Department  of  Defense  or  the 
U.S.  Government." 

The  acquisition  process.  The  con¬ 
gress  IS  also  committed  to  a  ma  )nr 
change.  the  american  public  need  to 
be  reassured  that  then  tax  dollars 
are  being  managed  in  a  competent, 
professional  manner.  While  there  is 
a  large  entrenched  bureaucracy  com¬ 
mitted  to  the  status  quo.  even  these 
groups  will  con.e  to  recognize  the 
inevitability  of  change.  It  is  the 
responsibility  of  the  professionals 
within  the  acquisition  workforce  to 
ensure  that  this  change  is  truly 
product ive . 

INTRODL’vlTION 

In  the  Presidential  election  of  lii? 
all  candidates  tan  on  the  platform  of 
change.  This  change  was  to  come  in  a 
variety  of  ways.  Each  candidate  had 
specific  and  general  recommendatioirs 
which  would  result  in  structural 
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changes  to  the  way  the  federal  gov¬ 
ernment  conducts  its  business  prac¬ 
tices. 

One  area  that  did  not  receive  atten¬ 
tion  during  the  campaign  was  the  way 
the  Federal  Government  contracts  with 
the  private  sector  for  goods  and 
services. 

Granted,  this  is  a  somewhat  esoteric 
subject  which  elicits  little  response 
from  the  average  American.  If  seri¬ 
ous  change  is  to  come  to  government , 
it  m.ust  come  not  only  in  what  the 
government  buys,  but  also  in  the 
manner  and  the  system  that  it  utiliz¬ 
es  to  accomplish  this  acquisition. 

There  is  a  general  consensus  that  the 
Clinton  administration  should  reduce 
the  level  of  expenditures  associated 
with  Defense.  The  nature  and  size  of 
the  threat  to  U.S.  security  has  un¬ 
dergone  a  substantial  change  over  the 
past  several  vears  and  it  is  only 
reasonable  to  accept  a  reallocation 
of  the  country's  resources.  This 
will  require  a  transition  from  De¬ 
fense  to  other  social  programs  (so¬ 
cial  in  this  context  to  include  the 
revitalization  of  the  country's  in¬ 
frastructure;  i.e.  roads,  bridges, 
transportation  etc.K  The  "Peace 
Dividend"  has  become  enmeshed  in  the 
prevailing  wisdom  of  the  day,  and 
most  Americans  anticipate  a  large 
savings  in  tax  dollars  from  the 
breakup  of  the  Soviet  Union  and  the 
conversion  of  Eastern  Europe.  Presi¬ 
dent  Clinton  has  emphasized  that  he 
will  utilize  this  savings  to  fund 
some  of  his  new  programs.  In  his 
book  he  promised  that  "Every  dollar 
we  save  by  downsizing  our  armed  forc¬ 
es  and  defense  industries  will  be 
reinvested  during  our  transition  to  a 
post-Cold  War  economy."^ 

As  funding  is  shifted,  it  is  impera¬ 
tive  that  Defense  related  industries 
be  accorded  the  opportunity  to  retool 
and  restructure  in  order  to  become 
competitive  in  alternative  venues. 


If  we  accept  the  premise  that  govern¬ 
mental  involvement  in  the  transition 
from  a  Defense  economy  to  a  non-De- 
fense  economy  is  imperative  to  avoid 
short-term  economic  instability,  then 
it  is  reasonable  to  ask  what  changes 
will  be  necessary  to  ensure  that  this 
transition  does  not  seriously  impact 
our  capabilities  to  protect  vital 
U.S.  strategic  interests. 

What  can  be  done  to  ensure  that  these 
interests  are  not  sacrificed?  One 
area  that  may  offer  potential  savings 
comes  in  the  form  of  restructuring 
the  DoD.  Services  are  evaluating 
mission  requirements  from  the  per¬ 
spective  of  substantial  downsizing. 
If  possible,  there  will  be  a  consoli¬ 
dation  of  missions,  with  a  concurrent 
reorganization  of  commands,  possibly 
even  extending  to  the  Service  level. 

President  Clinton  has  stated  that  he 
intends  to  cut  $60  Billion  from  the 
DoD  budget  over  the  first  five  vears 
of  his  Administrationls?) .  If  he  is 
truly  corrmiitted  to  the  funding  of 
those  ma3or  weapons  programs  that  he 
indicated  during  the  campaign,  then, 
as  I  see  it,  he  must  make  up  this 
savings  in  one  of  three  wavs. 

First,  he  can  withdraw  troops  cur¬ 
rently  stationed  overseas,  (Europe 
Korea  predominantly).  Second,  he  cm 
systematically  drain  the  O&M  and 
Personnel  accounts,  resulting  in 
massive  layoffs  of  defense  workers 
and.  most  likely,  a  return  to  the 
"hollow  forces"  that  existed  prior  to 
the  Reagan  buildup.  Finally,  he  can 
attempt,  through  reorganization  and 
attrition,  to  accomplish  more  with 
less  and  reduce  the  alleged  fraud, 
waste  and  abuse  that  many  ( f rom  the 
Grace  Commission  to  Ross  Perot)  have 
claimed  exists  in  the  DoD  bureaucra¬ 
cy. 

If  he  attempts  to  remove  troops  from 
overseas,  he  risks  the  possibility  of 
destabilizing  the  international  envi¬ 
ronment.  This  reduction  would  ulti- 


240 


"lately  reduce  our  ability  to  influ¬ 
ence  events  in  soiviu  of  our  most  vital 
strategic  interests.  In  the  event  of 
a  military  conflict  in  these  areas, 
he  would  be  forced  to  decide  between 
forsaking  allies,  and  reintroducing 
American  troops  -  possiblv  lisking 
American  casualties.  He  has  promised 
to  "call  on  our  allies  to  shoulder 
more  of  the  defense  burden".  He 
also  has  promised  to  "Meet  our  NATO 
responsibilities  in  Europe  with 
75,000  to  100,000  U.S.  troops,  rather 
than  the  150,000  troops  now  proposed 
by  George  Bush"^  Given  the 
increasing  volatility  of  this  region 
Mr  Clinton  has  legitimately  backed 
off  somewhat  from  his  initial 
position.  In  light  of  his  problems 
with  his  Vietnar;i  record,  this  would 
be  a  very  high  risk  scenario  if  it 
were  played  out  during  the  first  few 
months  of  his  presidency. 

The  risk  associated  with  reducing 
personnel  and  readiness  is  obvious. 
There  would  be  significant  resistance 
to  this  policy  if  it  results  in  the 
prolonged  unemployment  of  those  work¬ 
ers  who  faithfully  served  in  the 
defense  of  this  country  for  forty 
years.  Not  only  would  there  be  nega¬ 
tive  reaction  to  this  policy,  but 
there  could  be  severe  consequences 
upon  his  ability  to  initiate  new 
economic  programs.  Reduced  tax  reve¬ 
nue  accompanied  by  increased  unem¬ 
ployment  payments  would  mean  the  loss 
of  a  good  portion  of  the  peace  divi¬ 
dend  . 


countries  with  the  ability  to  project 
power  have  been  incapable  oi  unwill¬ 
ing  to  do  so  without  b'.S.  leadership. 
Nature  land  international  poliljcs' 
abhors  a  vacuurri.  When  we  disengage 
from  our  role  as  a  Hiaioi  power,  that 
role  will  be  assun.ed  bv  some  other 
country,  perhaps  one  whose  interests 
do  not  coinc:  ide  with  those  of  the 
U.S. 

One  ancillary  question  is  whether  we 
will  be  able,  or  willing  to  protect 
Israel  against  potential  adversaries 
in  the  middle  east?  Even  more  irnpoi- 
tant  is  the  matter  of  Israel 's  per¬ 
ception  of  that  question.  They  have 
demonstrated  a  credibility  to  use 
force  when  they  have  peiceived  their 
vital  interests  to  be  threatened. 
Given  the  current  buildup  of  arms  in 
the  middle  east,  it  is  most  certainly 
not  in  our  interest  to  have  a  con¬ 
frontation  between  Israel  and  her 
Arab  neighbors. 

If  we  have  learned  anytliing  from  the 
study  of  national  security,  it  is 
that  there  must  be  both  capability 
and  credibility  to  ensure  that  vital 
inter'ests  are  protected  and  aggres¬ 
sors  are  deterred  from  acting  based 
upon  the  basis  of  force  and  violenci'. 
If  we  reduce  our  force  posture  to  a 
"hollow  shell",  we  will  be  in  danger 
of  losing  both  the  capability  to 
respond,  along  with  the  cir'dibilily 
that  we  will  respond  to  those  situa¬ 
tions  which  we  may  deem  vital  to  our 
national  interests. 


Further,  if  certain  regimes  perceived 
some  advantage  from  their  ability  to 
use  military  force,  they  would  be 
more  prone  to  do  so  if  they  perceive 
the  major  powers  to  be  incapable  or 
unwilling  to  use  force  to  deter  them 
-  see  Iraq.  It  is  most  certain  that, 
in  my  view,  the  European  powers  will 
not  have  the  capability  (or  the  cred¬ 
ibility)  to  project  power,  even  to 
areas  of  vital  interest  to  both  them¬ 
selves  and  the  U.S.  Recent  history 
has  shown  that  most  of  the  European 


It  would  seem  apparent  then  that  the 
most  reasonable  course  of  action 
would  be  to  attempt  to  realize  sav¬ 
ings  by  increasing  efficiencies, 
while  concurrently  atteiiipting  to 
eliminate  much  of  the  fraud,  waste 
and  abuse  in  the  DoD. 

In  my  opinion,  one  area  that  offers 
potential  savings  through  efficien¬ 
cies  is  that  of  Do!)  restructuring. 
This  is  currently  in  practice  in  the 
private  sector  of  the  Defense  commu- 


nity  and  it  has  demonstrated  signifi¬ 
cant  results  in  terms  of  instant 
savings.  Companies  are  recognizing 
that  opportunities  for  Defense  busi¬ 
ness  will  be  reduced  in  the  future. 
They  are  consolidating  some  divisions 
and  eliminating  others.  Theoretical¬ 
ly,  this  will  result  in  a  more  effi¬ 
cient  and  streamlined  organization. 
One  theory  of  management  that  has 
gained  acceptance  for  assisting  DoD 
policymakers  in  this  reorganization 
is  that  of  Total  Quality  Management 
or  TQM. 

TQM  is  currently  the  official  manage¬ 
ment  initiative  of  the  DoD.  It  has 
been  inculcated  into  the  defense 
community  and  has  basically  been 
accepted  as  a  positive  vehicle  for 
change . 

In  accordance  with  the  principles  of 
Total  Quality  Management,  how  do  we 
accomplish  this  force  reduction  while 
continuing  to  ensure  that  national 
security  needs  are  accommodated? 

Total  Quality  stresses  the  principal 
of  continuous  process  improvement. 
One  process  which  offers  significant 
potential  for  savings  is  that  of 
weapons  acquisition.  The  structure 
that  has  served  this  country  so  ef¬ 
fectively  for  the  past  few  decades  is 
in  need  of  an  overhaul.  In  spite  of 
various  attempts  at  reform.,  there  are 
many  changes  that  may  be  implemented 
to  ensure  that  quality  products  are 
acquired  at  reasonable  cost  in  a 
timely  manner. 

There  was  some  recognition  that 
change  would  be  necessary  in  the  Air 
Force  through  the  creation  of  the  Air 
Force  Material  Command.  The  purpose 
of  this  consolidation  was  to  present 
"one  face"  to  the  customer  and  pro¬ 
vide  "cradle  to  grave"  support  for 
weapon  systems.  To  quote  the  Air 
Force  Chief  of  Staff  "About  five 
years  ago.  Air  Force  Systems  Command 
began  a  new  quest  for  total  quality 
leadership.  Our  customers  were 


demanding  it  and  our  people  had  many 
ideas  on  how  to  improve  our 
processes.  Under  the  Defense 
Management  Review,  these  commands 
continued  streamlining  and 
integrating  many  elements  of  our 
business,  moving  closer  to  a  single, 
uniform  acquisition  and  support  pro¬ 
cess  for  the  total  life  cycle.  As 
their  organizations  transitioned  and 
world  events  unfolded,  it  became 
clear  that  our  smaller,  more  flexible 
and  responsive  Air  Force  could  be 
better  served  by  a  single  command 
charged  with  supporting  all  the 
equipment  needs  of  the  warfighting 
commands.  The  single  supporting 
command  will  be  named  the  Air  Force 
Material  Command."  This  was  an 
admirable  attempt  by  the  Air  Force  to 
implement  change  in  an  orderly, 
structured  manner. 

Unfortunately,  to  some  observers,  it 
also  represented  an  attempt,  by  the 
Service,  to  avoid  major 
restructuring.  Restructuring  that  is 
critical  in  a  Clinton  Administration 
that  is  committed  to  change. 

I  believe  in  the  principles  espoused 
by  General  Yates  and  I  am  committed 
to  their  implementation.  I  believe 
that  the  Air  Force  simply  did  not  go 
far  enough  with  their  initiative. 
Had  the  OSD  taken  this  initiative  and 
implemented  it  across  the  services,  I 
believe  there  would  have  been  a 
significant  savings  realized  in  the 
cost  and  efficiency  of  the  DoD 
acquisition  svstem. 

If  we  take  the  concept  of  providing  a 
single  face  to  the  user,  and  extend 
it  to  the  different  processes  by 
which  each  individual  Service 
acquires  weapon  systems  we  must  see 
the  incongruity  herein.  The  Services 
should  be  able  to  provide  this  basic 
feature  to  all  Defense  customers 
rather  than  relegating  this  feature 
on  a  Service  only  basis. 
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BODY /TEXT 

What  is  the  mission  of  DoD 
Acquisition?  The  Air  Force  Material 
Command  has  stated  the  AFMC  mission 
is  to  "provide  an  Air  Force  that, 
with  the  other  armed  forces,  can: 

Preserve  the  peace  and 
security  of  the  United  States, 

Provide  for  its  defense, 

-  Support  national  policies. 
Implement  national 

objectives,  and 

Overcome  any  nation 
responsible  for  aggressive  acts  that 
imperil  its  peace  and  security  (sic) 
Teamed  with  the  Army,  Navy  and  Marine 
Corps,  the  Air  Force  is  prepared  to 
fight  and  win  any  war  if  deterrence 
fails. 

If  we  accept  the  principles  espoused 
by  this  mission,  what  are  some 
alternatives  in  process  improvements 
that  will  assist  the  Air  Force  (and 
the  DoD  in  general)  in  successful 
implementation  of  that  vision? 

First,  and  foremost,  1  believe  that  a 
major  overhaul  in  the  acquisition 
process  is  necessary  if  we  are  to 
continue  to  provide  the  quality 
products  necessary  to  insure  that  our 
fighting  men  and  women  will  have 
systems  with  technological 
superiority  built  into  them.  I  would 
submit  that  the  changes  to  date 
represent  an  evolutionary  approach 
toward  change,  whereas  I  believe  that 
a  revolutionary  approach  is  required. 

Why  is  revolutionary  change 
necessary?  And  why  is  it  necessary 
today?  Under  President  Clinton  the 
DoD  will  experience  a  rapid 
downsizing  that  is  unprecedented  in 
U.S.  history  outside  of  the 
conclusion  of  major  wars.  This 
change  is  inevitable  and  it  will  be 
substantial.  If  we  are  committed  to 
a  healthy  economy,  we  must  recognize 
that  it  is  ultimately  in  our  interest 
to  redirect  tax  revenues  from  defense 
to  other  national  programs  that  have 


higher  priority. 

Numerous  studies  have  been  conducted 
by  blue  ribbon  commissions  and 
Presidential  Task  Forces  into  ways  to 
improve  the  weapon  acquisition 
process.  A  commission  has  only 
recently  completed  their  evaluation 
into  the  regulatory  changes  that 
might  serve  to  expedite  the  process 
and  encourage  commercial  companies  to 
participate  in  DoD  business.  While 
these  studies  are  commendable,  they 
too  often  fail  to  grasp  the  magnitude 
of  the  problem  nor  are  they  able  to 
provide  the  "big  picture' 
recommendations  that  will  be 
necessary  in  the  coming  years. 

In  order  to  assess  the  need  for 
change  we  should  address  the  matter 
of  viability.  How  likely  is  it  that 
the  military  services  will  be  able  to 
buy  all  the  systeiris  that  they 
currently  have  in  process  given  the 
reduced  funding  that  will  be 
experienced  in  the  coming  years?  Is 
there  excessive  redundancy  within  the 
Services  and  between  the  Services 
both  in  regard  to  the  administration 
of  weapons  acquisition  and  the 
requirements  process  itself?  Or,  to 
put  it  more  simply,  do  we  have  a  need 
to  restructure  the  system,  given  the 
reduction  in  threat  and  the  reduction 
in  dollars? 

I  believe  the  answer  to  these 
questions  is  yes  and  I  also  believe 
that  we  can  not  afford  to  wait  until 
It  becomes  necessary  to  cancel 
weapons  programs  due  to  lack  of  funds 
before  we  acknowledge  this  reality. 
The  Department  of  Defense  must  take 
the  initiative  to  forestall  this 
eventuality.  In  order  to  accomplish 
this,  major  changes  must  take  place 
within  the  DoD  acquisition  system. 

What  are  these  changes,  and  how  will 
they  be  implemented?  There  are  a 
number  of  changes  currently  in 
process,  however  additional, 
substantive  changes  must  be 
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implemented,  and  soon. 

One  major  chan^.e  that  could  be 
implemented  is  the  creation  of  an 
Acquisition  Service  within  the 
Department  of  Defense.  This  Service 
would  be  responsible  for  the 
acquisition  of  all  weapon  systems  by 
the  DoD.  This  approach  has  been 
advocated  bv  some  in  the  defense 
corrirnunitv .  Mi  Bernard  Schwartz,  CEO 
of  Loral  Corp.  in  a  speech  to  a 
conference  on  the  Defense  industrial 
base  advocated  "The  elimination  of 
redundancy  in  nilitarv  procurement  by 
consolidating  the  four  service 
acquisition  bureaucracies  into  one"** 

The  creation  of  this  Service  would  be 
entirely  consistent  with  the 
reorganization  that  many  DoD 
contractors  are  undergoing.  As  I 
discussed  above,  the  consolidation 
and  elimination  of  redundant 
functional  organizations  within  DoD 
corporations  is  aimed  at  streamlining 
these  companies  and  ensuring  their 
competitiveness  in  a  shrinking 
market.  These  restructuring  are  also 
aimed  at  transitioning  from  defense 
related  business  to  other  non-defense 
business . 

Why  shouldn't  the  DoD  take  a  page  out 
of  the  private  sector  and  restructure 
into  a  more  efficient  organization? 
Business  as  usual  is  no  longer 
acceptable  given  the  new  reality. 

The  Acquisition  Service  would 
effectively  be  a  hybrid  between  the 
Civil  Service  and  the  Military 
Service.  In  my  opinion,  it  is 
critical  that  we  remove  the  military 
from  the  acquisition  business.  This 
needs  to  be  done  not  because  they  are 
not  extremely  capable,  dedicated, 
hard-working  acquisition  employees. 
Rather,  it  is  necessary  because  it  is 
simply  not  fair  to  have  them  serve 
two  masters.  They  must  decide,  in 
many  cases,  whether  it  would  be  more 
beneficial  to  their  career  to  fulfill 
military  or  acquisition  career 


responsibilities.  By  this  1  mean 
that  there  are  multiple  demands  upon 
their  time.  They  cuirentlv  must  meet 
military  standards  such  as 
professional  military  education  which 
may  make  them  a  better  military 
officer,  but  most  certainly  adds 
little  to  their  business  acumen. 
They  also  must  meet  professional 
acquisition  requireiuents  such  as  the 
Defense  Systems  Management  College  or 
the  Air  Force  Institute  of  Technology 
lAFIT).  Many  officers  are  foiced  to 
choose  between  these  two  options.  In 
an  Acquisition  Service,  professional 
development  would  have  the  highest 
priority  in  an  individual's  careei 
development . 

The  Acquisition  Service  would  also 
help  avoid  much  of  the  inter-service 
rivalry  which  can  result  in  redundant 
weapon  systems.  It  would  be  sniilar 
to  the  type  of  system  that  was  recom¬ 
mended  by  the  Assistant  Secretary  of 
Defense  for  Acquisition  and  Logistics 
in  his  1985  report,  an  independent 
service  that  woulii  report  directly  to 
the  Secretary  of  Defense. 

What  are  the  implications  of  such  a 
major  restructuring  of  the  acquisi¬ 
tion  system?  The  concept  of  an 
Acquisition  Service,  independent  of 
the  military  services  (Army,  Navy  & 
Air  Force),  is  somewhat  revolution¬ 
ary.  The  concept  of  eliminating  the 
military  from  within  this  service  is 
even  m.ore  so. 

What  unique  qualities  do  the  military 
bring  to  the  procurement  process? 
Critics  of  the  concept  of  eliminating 
the  military  from  the  acquisition 
business  most  often  point  out  that 
the  military,  being  the  eventual 
users  of  any  weapons  that  are  pro¬ 
cured,  are  best  able  to  provide  in¬ 
sight  as  to  the  concerns  of  the  figh¬ 
ting  men.  This  argument  loses  much 
credibility  when  subjected  to  the 
light  of  reason.  If  an  officer,  at 
the  Program  Manager  level,  actually 
has  experience  in  the  use  of  a  weapon 
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system,  it  is  most  likely  that  the 
weapon  system  is  obsolete.  The  fact 
that  a  General  served  on  a  Minuteman 
Missile  crew  twenty  years  ago  does 
not  provide  him  with  additional 
credibility  when  discussing  the 
unique  features  of  a  Peacekeeper  or 
Small  ICBM  weapon  system. 

Weapon  system  technology  is  advancing 
at  a  more  rapid  pace  today  than  ever 
before,  and  will  only  accelerate  in 
the  future.  This  means  that  problems 
that  were  experienced  in  the  past  may 
have  no  relevance  to  future  systems. 
In  any  event.  if  operational 
experience  is  determined  to  be  an 
essential  element  of  a  competent  ac¬ 
quisition  specialist,  then  former 
military  officers  with  weapon  system 
experience  could  be  hired  after  com¬ 
pletion  of  then  tour  of  duty,  or 
acquisition  service  individuals  could 
be  permitted  to  accomplish  a  tour  of 
active  military  duty  with  a 
particular  weapon  system.  The 
complicated  requirements  associated 
with  the  acquisition  system  dictates 
that  individuals  must  be  trained  in  a 
broad  spectrum  of  disciplines,  from 
entry  into  the  military,  through  the 
latter  stages  of  the  officers  career. 

The  Packard  Commission  recognized  the 
criticality  of  this  requirement,  and 
recommended  that  military  officers 
proceed  along  a  "well  defined 
acquisition  career  program"^  toward 
improving  the  qualifications  and 
training  of  Program  Manageis.  The 
Air  Force  has  initiated  a  development 
training  program  for  acquisition 
professionals  that  includes  career 
development  assignments,  training  and 
education  with  the  ultimate  goal  of 
professionalizing  the  Acquisition 
workforce . 

This  program  (The  Acquisition 
Professional  Development  Program  or 
APDP)  combines  the  "three  pillars"  of 
Experience,  Training  and  Academics  to 
guide  managers  through  successive 
levels  of  development.  While  this  is 


indeed  a  commendable  goal,  it  has 
little  bearing  upon  the  fundamental 
question  as  to  the  need  for  the 
military  in  the  acquisition  business. 
If  it  is  sensible  to  develop  a  future 
executive  through  a  structured,  long¬ 
term  program  (and  few  would  argue 
with  this  position),  then  perhaps  the 
process  would  best  be  served  by 
eliminating  the  military  requirement 
in  its  entirety. 

This  premise  is  based  primarily  upon 
two  main  considerations.  The  first 
is  that  the  m.ost  productive  period 
for  a  business  executive  is  the 
period  between  his  40's  to  bO's.  It 
is  during  this  period  that  he 
realizes  the  results  of  his  formative 
period.  The  military  services 
traditionally  lose  a  good  many  of 
their  more  successful  officers  during 
this  period.  Only  a  relatively  small 
percentage  of  individuals  receive 
promotion  past  the  rank  of  Lieutenant 
Colonel . 

While  it  is  true  that  those  indivi¬ 
duals  who  remain  are  truly  outstand¬ 
ing,  it  is  a  great  waste  of  potential 
to  lose  those  who  are  not  promoted  to 
the  rank  of  Colonel.  The  rest  of  the 
highly  productive,  successful 
acquisition  experts  are  allowed,  even 
encouraged,  to  leave  the  military. 
The  fact  that  such  a  large  percentage 
of  these  individuals  go  to  work  for 
defense  contractors,  in  senior  level 
positions,  merely  underscores  the 
fact  that  valuable  resources  are 
being  wasted  under  the  current 
system. 

It  may  be  perfectly  reasonable  to 
have  a  retirement  program  in  effect 
that  encourages  individuals  in  their 
forties  to  retire  from  military 
service.  It  is  not  reasonable  to 
have  acquisition  specialists  retiring 
at  such  a  relatively  earlv  age.  For 
a  "warrior"  the  forties  represents 
the  beginning  of  the  decline  of 
physical  abilities.  It  is  ludicrous, 
however,  to  lose  an  individual  whose 
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primary  responsibility  is  that  of 
business  management  (i.e.  an  intel¬ 
lectual  career)  at  that  relatively 
earlv  stage  in  his  career. 

The  military  relinquishes  the 
benefits  of  the  experience  and 
ability  of  highly  skilled,  dedicated 
individuals  who  have  only  recently 
reached  the  zenith  of  their  produc¬ 
tive  capability.  It  is  perhaps 
useful  to  compare  the  career  tracks 
of  the  professional  military  acquisi¬ 
tion  executive  with  that  of  execu¬ 
tives  within  the  private  aerospace 
community.  During  the  age  at  which 
military  executives  are  "retired” 
from  government  service,  aerospace 
executives  are  still  in  the  develop¬ 
mental  stages  of  their  careers.  The 
most  productive  period  for  executives 
in  industry  is  the  AS-bO  year  age 
group.  It  seems  inconsistent  at 
best,  and  highly  wasteful  at  worst, 
to  allow  the  military  system  to  drive 
out  some  of  the  most  respected 
members  of  their  profession  who  still 
have  many  productive  years  remaining. 
The  proof  of  this  productivity  is  the 
willingness  of  private  aerospace 
contractors  to  offer  them  important 
positions  within  their  industry. 

The  second  consideration  relates  to 
the  military's  "up  or  out"  system. 
Military  members  are  often  promoted 
out  of  a  position  in  which  they  are 
most  productive  and  required  to 
accept  a  position  in  a  management 
capacity.  While  this  may  be  reason¬ 
able  when  dealing  with  those  indivi¬ 
duals  in  the  fighting  branches  (and 
some  would  argue  that  it  is  not  a 
reasonable  approach  even  there),  it 
is  most  unfortunate  that  such  a 
policy  would  exist  in  the  acquisition 
field.  There  are  any  number  of 
individuals  who  would  only  ask  that 
they  be  allowed  to  remain  in  their 
current  position.  The  research 
engineer  and  the  theoretical 
scientist  who  happen  to  be  officers 
in  the  military  are  required  to  move 
"up  the  ladder"  to  administrative  and 


management  positions  if  they  are  to 
remain  in  the  military  (with  only 
several  exceptions).  This  places  the 
individual  in  the  onerous  position  of 
having  to  choose  between  a  career  and 
a  job.  As  the  military  officer 
advances  in  grade,  from  company  grade 
to  field  grade,  his/her  respon¬ 
sibilities  become  more  management 
oriented . 

This  type  of  choice  would  not  be 
required  in  an  Acquisition  Service. 
Some  would  argue  that  the  individual 
has  the  option  of  working  for  the 
civil  service.  Unfortunately,  this 
option  has  severe  limitations  as 
well.  The  key  point  is  that  a 
significant  loss  of  productivity  is 
tied  to  the  individual's  association 
with  the  military  system  and  the 
responsibilities  associated  therein, 
rather  than  to  the  acquisition 
system. 

Having  articulated  the  two  major 
criticisms  of  the  military  system,  I 
would  be  remiss  if  I  do  not 
articulate  some  of  the  benefits 
associated  with  having  military  in¬ 
dividuals  involved  in  the  acquisition 
process.  The  first  benefit,  would  be 
the  broad-based,  highly  motivated, 
professional  individuals  who  enlist 
in  the  military  to  serve  their 
country.  Would  the  commensurate 
level  of  quality  individuals  be 
available  to  an  acquisition  service? 
Could  an  acquisition  service  attract 
and  retain  individuals  who  are  as 
talented  and  dedicated  as  those  who 
currently  hold  down  these  assign¬ 
ments?  Could  we  expect  the  blue- 
suiters  to  trade  them  in  for  business 
suits  or  would  the  government  lose 
these  individuals  to  the  private 
sector? 

In  the  course  of  discussing  the 
premise  that  we  need  to  get  the 
military  out  of  the  acquisition 
business,  there  is  significant 
concern  on  the  part  of  some  in  the 
military  that  civil  service  employees 
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are  neither  as  dedicated,  nor  as 
talented  as  their  contemporaries  in 
uniformed  service.  They  do  not 
believe  that  civilians  would  be  as 
motivated,  nor  as  willing  to  make 
sacrifices,  i.e.  mobility,  that  are 
required  to  ensure  that  the 
acquisition  system.  operates 
effectively. 

1  believe  that  there  are  many  flaws 
in  the  way  that  the  civil  service 
personnel  system  is  currently 
structured.  An  Acquisition  Service, 
with  a  system  of  rewards  and  benefits 
similar  to  those  of  the  military, 
would  attract  m.ore  qualified  indivi¬ 
duals  than  does  the  current  system. 
This  contention  is  based  upon  several 
basic  assumptions,  the  first  would  be 
that  there  would  be  a  revision  to  the 
personnel  system  to  allow  comparable 
piofessions  for  acquisition  service 
erriployees.  This  would  allow  the 
services  to  compete  for  the  better 
educated.  more  highly  motivated 
individuals  who.  it  has  been  argued, 
currently  come  from  the  military 
:..e;:,be i of  tin;  acquisition  workforce. 
In  this  context.  an  A.ssistant 
Secretary  of  Defense  for  Acquisition 
and  Logistics  has  proposed  that: 

"a  fresh  approach  is 
necessary  to  assure  that 
the  DoD  is  able  to 
maintain  a  first  rate 
piofessional  '■''quis  1 1  ion 
and  logistics  workforce. 

We  lack  a  cadre  of 
seasoned,  well-rounded, 
technically  oi  rented 
acquisition  profes¬ 
sionals.  These  people 
are  the  kev  to  improving 
t  h  e  DoD  s  V  s  t  e  m  s 
acquisit  lOii  proces,--. 

While  no  /in.ount  of 
reorgarll^at  ion  will,  bv 
Itself,  solve  these  pru- 
blerris.  a  svsi^m  which 
lesults  in  cleailv 
defined  lines  of 
authority.  tesporisi- 


bility,  and 
accountability  and 
manned  by  experienced, 
high  tech  oriented, 
professionals  dedicated 
to  making  the  system 
work  has  a  far  better 
chance  of  success...  A 
true  service  Corps  with 
dedicated  professionals 
is  a  necessity. 
Management  of  the  Corps, 
in  the  main  must  be  by 
civilians  rather  than 
military  personnel  who 
rotate  out  after  a 
number  of  years. 
Experience  and  ability 
must  be  the  key  determi¬ 
nants  for  advaiicement . 
as  well  as  more 
discipline  in  carrying 
out  the  objectives  of 
the  Office  of  the 
Secretary  of  De¬ 
fense... We  do  not  need 
more  people — but  we  do 
need  superior.  better 
qualified  people.  To 
attract  these  people  we 
need  changes  in  pay 
structures.  rotational 
programs,  training  and 

more  flexible  personnel 
8 

procedures." 

While  It  is  true  that  this  theme  (the 
need  for  a  more  professional 
workforce)  has  been  a  consistent  one 
throughout  the  many  commissions  on 
procurement  reform.  i.e.  Grace 
Commission,  Hoover  Commissions  etc., 
it  IS  nonetheless  true  that  given  the 
new  realities  of  a  sizeable  force 
reduction,  a  consensus  is  developing 
toward  the  need  for  a  revitaiizat jon 
of  this  workforce.  It  .,;ust  be  made 
quite  deal  that  the  elimination  of 
the  militarv  from  the  acquisition 
workforce  must  be  accomplished  in 
conjunction  with  the  overhaul  of  the 
acquisition  personnel  system  or  the 
result  would  most  likely  be  the 
exchange  of  one  set  of  problems  for 
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those  of  another. 


incompetence . 


I 


The  Packard  Commission  and  the 
Defense  Management  Review  both 
recognized  the  need  for  significant 
personnel  restructuring  in  their 
reports  to  the  President. 

"One  extrerrielv  important 
means  to  improve  the 
acquisition  workforce 
IS  to  establish  an 
alternative  personnel 
management  system 
permitting  greater 
flexibility  with  respect 
to  the  status,  pav  and 
qualifications  of 
civilian  employees." 

All  experimental  personnel  system  that 
was  evaluated  was  the  China  Lake 
project.  Results  from  that  project 
indicated  that  there  was  an  increase 
in  productivity  along  with  a  higher 
degree  of  job  satisfaction.  The 
positive  feedback  from  this  program 
demonstrates  that  there  truly  is  a 
dedicated  corps  of  acquisition 
specialists  who  would  like  to  see  the 
personnel  system  overhauled  to  reward 
productivity  and  eliminate  the 
deadwood.  We  need  to  overhaul  the 
civil  service  personnel  system 
through  use  of  the  stick  and  the 
carrot.  The  stick  to  eliminate  the 
nonprodiicei  s .  and  the  carrot  to 
leward  tire  ovc’ r -achieve  i  s .  Super¬ 
visors  and  managers  are  continually 
frustrated  bv  the  lack  of  effective 
remedies  against  non-productive 
e;..pluvees.  The  impact  of  these 
indivitinals  upon  the  ,:iorale  of  the 
nr  garrizat  iotr  cannot  be  overstated. 
It  is  unfortunately  all  too  true  that 
incompetent  and  lazv  individuals  are 
often  relegated  to  ’:enial  duties 
while  still  receiving  the  benefits  of 
seniority  and  pay  increases  that  are 
due  to  the  luoie  co;.,petent  coworkers. 
In  some  cases  individeals  have  even 
been  ptoTi.oted  in  order  to  gel  them 
out  ot  a  particular  position,  where 
they  have  d  innons  1 1  a  t  ed  an 


If  there  is  to  be  a  stick,  then  it  is 
also  appropriate  that  there  be  a 
carrot.  Rather  than  reward  an 
employee  for  time  in  grade,  i.e. 
solely  on  seniority,  it  would  be  more 
productive  to  reward  him  for  the 
degree  of  success  that  he/she  may 
have  in  their  position  in  conjunction 
with  the  degree  of  difficulty  of  that 
position.  Thus,  an  engineer  on  a 
particularly  complicated  weapon 
system  would  receive  a  higher  pay 
than  one  on  a  less  complex  one,  even 
if  they  do  comparable  work. 

It  has  been  argued  that  the  govern¬ 
ment  can  not  compete  'ith  the  private 
sector  for  the  lii.^ted  number  of 
skilled  individuals  in  the  weapons 
acquisition  field.  While  there  may 
be  an  element  of  validity  in  this 
position,  the  services  have  tradi¬ 
tionally  awarded  contracts  to  the 
private  sector  for  engineering 
services  (for  example  the  Air  Force 
has  contracts  with  TRW  for  Systems 
Engineering).  Theie  is  no  reason  to 
believe  that  these  individuals  would 
not  work  for  an  Acquisition  Seivice 
if  it  were  able  to  offer  comparable 
wages  and  benefits.  If  this  were  to 
be  a  problem,  many  of  the  program 
engineering  requirements  could  be 
contracted  out  to  systems  engineering 
firms  such  as  TRW.  It  is  most 
likely,  however,  that  given  the  large 
reductions  associated  with  the  DoD 
downsizing,  there  will  be  a  highly 
qualified  and  motivated  workforce 
available  to  draw  from. 

If  the  military  is  to  be  eliminated 
from  the  acquisition  process,  then  it 
would  seem  prudent  for  the  new 
Acquisilion  Service  to  incot  por  at  t' 
some  of  the  n.ilitary's  more  forward 
thinking  programs.  The  ROTC  and  OTS 
programs  provide  the  military  with 
future  leaders.  They  are  extrernelv 
valuable  riir'ans  of  ensuring  a 
continuous  suppiv  of  leadershiji  for 
the  services.  ROTC  utilizes  the 
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lesources  of  collo^.e  universities  to 
supply  future  leadeis  with  both  an 
education  and  an  opportunity  to 
expei lence  the  military  environment 
prior  to  commissioninj?,.  The 

Acquisition  Corps  should  consider  a 
training  program  at  the  college  level 
which  would  provide  a  similar 
opportunity  for  future  acquisition 
leadeis.  In  exchange  for  joining  the 
K'i|uisition  service,  and  incurring  a 
foui  year  commitment,  college 
students  in  appropriate  curricula 
would  bf'  offered  scholarships  and 
training  in  the  acquisition 
prolession.  This  would  include 
spec  ific  ac(|uisi  t  ion  training  as  well 
as  a  snmriiei  camp  at  a  program  office 
oi  acquis  1 1 ion  headquarters.  There 
aie  raiiv.  puss  i  b  i  i  i  1 1  f's  for  this  type 
of  program;  however,  the  basic 
premise  would  be  that  executive 
intern  candidates  fron;  this  program 
would  have  .in  acquisition  experience 
lather  than  a  military  one.  Similar 
inogi.iriis  at  advanced  management 
levels  crnild  be  implemented  with 
associated  commitment  to  government 

sei  \  K  . 

Uliile  the  two  n.ajot  reasons  for  the 
el  in, mat  ion  ot'  the  military  have  beiMi 
discussed.  tln'ie  aie  addition.!! 
piolilems  that  are  a.-.sociated  with 
hivin;',  the  ;..jlltar\  in  the  acquisi- 
t  ion  busivie-s.  One  of  thi’se  is  the 
I  i  111  I  I  I  c  I  I  n;'.  demand  upon  the  iiiilitary 
.till',  fill  111;-  t  I  I  I  n  1  nr.  and  ediii  a- 
l  I  nil .  M  I  1  1 1  1 1  \  III  1  1  ce  I  s  aie  hi  ghl  y 
ein  am.  .i;-,nil  to  iliVt'  P  lO  f  ^■S,'-.  I  Olia  1 
'1 1  1  1 1  1 1 KdiK.it  mn  '  I’MK  '  completed  at 
■per  itii  priints  in  t  InM  i  carer'r.s. 
llii-  education  'loi  the  An  Force ) 
c  fni:-.  1 ,  . I  ol  h.isic  I’MK  '  I'l  of  essimia  1 
Ml  lit  II  \  IdiiiMtion  e,g.  Squ.idrrui 
OtiK'ei-.  Si  'mull.  1  nt  e  r  .led  la  t  e  level 
K'li  '1  eiViie  .^t.ilf  crilleg.e  e.g,,. 
All  l  ie  :  and  iiid  Slalfi  ,ind  a  .-.eincii 
level  KMI  'a  >ei\ue  wai  college  (ii 
equivalent,  e.;, ..  All  W.ir  tolleg.e  ix 
the  lndn:-.liial  lollege  of  the  An..ed 
rorces  ICAK'.  Additional  PMF 

miclufle-.  Aimed  liiices  Stalf  I’nllegr*. 
ind  the  N.tv.ii  f’o-^  t  g  (  adn.i  I  e  .School. 


While  these  courses  offer  much  that 
IS  useful  to  an  officer  in  the 
acquisition  career  field,  it  is  a 
fact  that  the  curriculum  is  tailored 
for  military  officers,  and  as  such 
contains  much  inf  oimiation  that  is 
essentially  irrelevant  to  the 
majority  of  officers  in  the  acquisi¬ 
tion  business.  The  related  material 
could  be  addressed  in  educational 
training  that  would  be  accomplished 
in  conjunction  with  acquisition 
related  course  work  (i.e.  at  the 
Defense  Systems  Management  College). 

Another  problem  that  is  associated 
with  the  military  is  that  of  rank 
sensitivity.  The  military,  being 
essentially  an  authoritarian  society 
with  the  hierarchy  cle.  ly  defined  liv 
the  rank  on  an  individual's  sleeve  or 
shoulder,  is  not  often  receptive  to 
dissenting  opinions.  The  head  of  an 
independent  acquisition  service  would 
be  more  likely  tc  respond  to  the 
produceabi  1  ity  and  need  for  a 
.specific  weapon  system  without  the 
pressures  that  are  inherent  to  a 
iiiilil.ny  officer  under  tin-  milllaiv 
.hain  of  conin.and. 

This  IS  not  to  sav  that  all  political 
leverage  would  be  eliminated  from  the 
acqinsil  ion  iiroce.-s.  The  reality  of 
the  .situation  is  that  there  are 
political  forces  and  piessnies  on 
those  inflnentL.i'  in  the  acquisition 
decision  i.i.ikinr;  .stiintuie.  I'.iven  tln- 
rmormons  sums  that  are  requited  to 
develop  and  field  .i  n.aior  weapon 
svsli'r.i  this  IS  1  in-v  1 1  .\b  I  e ,  By 

e  1 1  ni  1  n.r  ling  the  n:  i  1  1 1  a  i  v  f  r  om  I  he 
acquisition  system,  one  set  of 
politic.rl  pilot  It  les  will  he 

el  iii.inil  ed  .  The  competition  between 
the  Military  .Service.s  will  not  go 
aw.iv  vcitli  ilie  cii  itlon  of  .in  ArquiSl- 
I  inn  Corps.  It  should  however 

provide  a  les.s  bi.ised  opinion 
regarding,  the  capabilities  of 
specific  weapons.  The  decision  as  to 
how  lesonrce.s  aie  to  Ire  allocated 
will  not  change  mat erial 1 v .  The 
viability  111  Ih.rl  .system  is  not  the 
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subject  of  this  paper.  The  indepen¬ 
dent  nature  of  a  non-military 
acquisition  corps  in  theory  would 
help  to  override  some  of  the  inter¬ 
service  rivalries  that  have  been 
known  to  create  redundant  systems  due 
to  one  service's  unwillingness  to  buy 
another's  system. 

Another  concern  regarding  the  use  of 
military  in  the  current  system 
relates  to  the  lack  of  cohesive 
policy  between  the  services  regarding 
the  roles  and  responsibilities  of  the 
military  personnel  within  the 
acquisition  process.  Each  service 
has  its  own  unique  style  regarding 
the  method  that  they  buy  weapons. 
This  often  creates  difficulties  for 
defense  contractors  who  have 
experience  with  one  branch,  when  they 
wish  to  compete  for  programs  offered 
bv  another  service. 

The  Air  Force  has  military  officers 
assigned  as  Program  Managers  of 
System  Program  Offices  with  project 
engineers  (usually  Lieutenants  or 
taptdins»  assigned  to  subsystem 
responsibility.  Several  layers  of 
management  coordinate  the  total 
program  acquisition,  with  a  senior 
officer  (usually  a  one  or  two  star 
General  Officer)  acting  as  Program 
Executive  Officer.  Acting  as  Systems 
Engineering  support  may  be  a  private 
contractor  (such  as  TRW).  The  army 
uses  civilians  as  the  heads  of  their 
directorates  and  also  utilizes 
civilian  engineers  in  a  project 
engineer  capacity.  The  Navy  normally 
has  a  small  program  office  staff  of 
riilitary  and  civilians  but  contracts 
out  to  the  private  sector  for  a 
svste:.:s  integiatoi  who  will 
accomplish  much  of  the  development 
that  the  otliei  sr'ivices  do  them¬ 
selves.  The  different  approaches  to 
weapon  syster,  acquisition  has  created 
uncertainties  when  contractors  are 
leqiined  to  participate  in  different 
service ' s  programs . 

I'onsislencv  of  approach  toward  weapon 


system  acquisition  would  go  a 
significant  way  toward  improving 
contractor/government  relations. 
This  would  be  possible  through  a 
central  civilian  acquisition  service 
because  while  the  terminology  of 
position  descriptions  may  vary  from 
Service  to  Service  the  essential 
position  characteristics  remain  very 
much  the  same . 

Interestingly,  the  promotion  system 
of  the  military  is  seen  to  be  both  a 
positive  and  a  negative  considera¬ 
tion.  While  the  system  is  capable  of 
weeding  out  the  incompetent,  and  does 
provide  a  greater  degree  of  flexibil¬ 
ity  for  supervisors,  promotions  are 
based  on  evaluation  reports  by 
supervisors,  which  have  required 
overhaul  every  few  years  due  to  OPR 
inflation.  The  competitive  nature  of 
the  rating  system  serves  to  motivate 
some  individuals,  however  it  also 
tends  to  discourage  initiative  and 
leads  to  a  mind-set  of  following  the 
path  of  least  resistance.  It  also 
tends  to  discourage  risk  taking  and 
iniliative,  two  valuable  commodities 
in  the  new  period  of  change. 

The  rotational  nature  of  the  military 
also  provokes  opposing  viewpoints. 
One  view  holds  that  I  he  military,  by 
the  nature  of  their  mobility,  are 
better  able  to  bring  a  comprehensive 
perspective  to  the  acquisition 
process.  The  opposing  view  states 
that  the  military  become  jack-of-all- 
trades  and  master  of  none.  Rotational 
assignments  often  result  iu  the  loss 
of  corporate  memory  and  create 
program  instability. 

There  is  an  inherent  instability 
associated  with  having  a  Program 
Manager  evaluated  on  tile  b.isis  of 
short-teimi  results.  The  incentive  rs 
to  forsake  long-term  creative 
solutions  that  have  more  up-front 
costs  because  programs  are  most 
vulnerable  to  cancellation  during  the 
early  stages  of  development.  Evalua¬ 
tions  are  based  upon  the  present  and 
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most  recent  past.  Thus,  a  manager 
who  comes  in  to  solve  a  problem  is 
given  better  marks  than  one  who 
avoids  the  problem  in  the  first  place 
through  good  management. 

The  DMR  recognized  this  system 
weakness  and  recommended  that  Program 
Managers  be  evaluated  upon  the  long 
term  success  or  failure  of  a  weapon 
system.  To  provide  effective 
evaluation  of  an  acquisition  manager, 
there  should  be  more  of  the  cradle- 
to-grave  approach.  If  he  is  given 
the  responsibility  for  correcting 
future  problems,  the  program  manager 
would  be  more  receptive  to  short-term 
losses  if  they  would  result  in  long¬ 
term  gains.  The  corollary  that 
incentives  must  be  provided  to 
motivate  quality  individuals  toward  a 
career  in  DoD  acquisition  is  obvious. 
A  related  concern  would  be  the 
necessity  to  reward  program  managers 
even  if  they  are  the  bearers  of  bad 
tidings,  i.e.  if  programs  overrun  or 
prove  impractical,  early  recognition 
should  be  rewarded,  not  punished. 

Given  the  visibility  of  the  revolving 
door  situation,  does  it  make  sense  to 
expect  individuals  to  work  for  the 
government  at  reduced  salaries  with 
no  expectation  of  eventually  going  to 
work  for  the  private  sector?  The 
Services  and  the  DoD  are  aware  of  the 
increased  support  within  the  Congress 
for  a  change  in  the  way  weapon 
systems  are  acquired.  The  DMR 
represented  lat  least  in  part)  a 
recognition  that  the  system  was  in 
need  of  a  change. 

linfortunately ,  the  DMR  doesn't  go  far 
enough  in  restructuring  the  weapon 
system  process.  The  Progiam 
Executive  Officer  concept  has  created 
an  additional  set  of  problems  and 
concerns.  Rather  than  simplifying 
the  organizational  structure,  it  has 
resulted  in  a  convoluted  system  that 
often  seems  to  incorporate  the  worst 
of  both  worlds.  The  intent  was 
streamlining,  the  result  was  a 


somewhat  disorganized  system  that 
creates  uncertainty  at  the  working 
level  (i.e.  Program  Office)  with 
regard  to  chain  of  command  and 
program  responsibility. 

CONCLUSION 

The  Defense  Acquisition  Workforce 
Improvement  Act  attempts  to  reconcile 
some  of  these  problems  however,  it  is 
a  band-aid  approach.  The  band-aid 
approach  will  not  resolve  the  need 
for  major  structural  changes  in  the 
Clinton  Administration's  DoD.  The 
elimination  of  the  military  will  not 
resolve  all  of  the  defects  in  the 
system.  It  will,  I  believe,  in 
conjunction  with  the  other 
recommendations  identified  herein, 
serve  to  improve  the  system  and 
eliminate  some  of  the  problems 
associated  therein,  the  conclusions 
of  this  paper  are  as  follows: 

1.  Remove  the  military  from  the 
acquisition  business.  Offer  those 
members  of  the  uniformed  services 
currently  on  active  duty  the  option 
of  transitioning  into  an  acquisition 
service  or  accepting  alternative 
career  positions  within  the  military. 
Equity  would  dictate  that  those  who 
choose  to  remain  in  the  Acquisition 
Service  be  given  credit  toward 
retirement  for  their  time  in  the 
military.  Perhaps  a  graduated  plan 
that  would  allow  those  individuals 
with  significant  time  to  retire 
closer  to  the  current  20  year  system. 

2.  Transition  those  individuals 
within  the  civil  service  into  the 
acquisition  service.  There  would  be 
a  separate  set  of  personnel  regula¬ 
tions,  perhaps  similar  to  the 
excepted  service.  There  would  also 
be  a  new  pay  and  incentives  struc¬ 
ture.  Reference  the  China  Lake 
project  for  recommendations  along 
this  vein.  Additional  training  would 
be  mandatory  for  those  lacking  the 
minimum  mandatory  acquisition 
courses . 
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3.  Iniliate  those  actions  required 
to  inipleiiput  an  Acquisition  Executive 
Intern  Training  Program  at  selected 
Universities  to  develop  future 
Acquisition  Executives. 

4.  Standardize  an  executive  develop- 
r:;ent  training  program  for  use  in  the 
Acquisi tion  service,  adopting  the 
best  ot  each  of  thi"  services  current 
pi  ogi  a.‘  s . 

5 .  Consider  the  possibility  of 
consolidating  r.,anv  of  the  Systems 
Enginepi  ing,  conti  actor  functions  into 
the  acquisition  Service.  This  would 
necessitate  a  pav  scale  commensurate 
witli  the  Contractors  who  provide 
these  seivices  <i.e.  TRW.  Aerospace), 
however,  given  the  current  employment 
outlook.  It  would  not  seem  unreason¬ 
able  to  bring  the  required  expertise 
on  hoard  at  .i  rather  large  reduction 
in  cost  to  the  DoD. 

The  hot  tor,,  line  is  to  remove  the 
:;iilitarv  r  ror:.  the  process  in  all 
capacities  except  that  for  which  they 
are  most  ideal  Iv  •' u  i  (  ed  ,  and  that  is 
as  the  User  ot  the  final  product. 
This  will  onlv  prove  effect  ive  in  an 
inilependent  Acquisition  Service.  It 
will  also  onlv  be  rd' f  r-c  t  ivc'  if  the 
changes  to  the  civil  service  are 
accorup  1  ished  in  con  j  uric  t  ion  with  the 
creation  of  the  .Acquisition  Service. 

It  is  \  1 1  .r  II  .  I  .iipur  I -int  to  recognize 
that  this  i  eco’iii.iiMid.ii  inn  in  no  w.iv 
lefler  t.s  upon  the  l  ine  |oh  that  the 
i.ilitaiv  provide  to  the  current 
a(((uisition  svsfeii;.  Ihf'Sf’  chaitge.s 
will  not  a’ieliorate  anv  of  the  other 
.i-i|Oi  concerns  that  if  late  to  the  wav 
the  liol)  coikIoc  I  s  its  acqti  i  s  1 1  i  on 
lius  I  ness  .  hndget  t  n  rtni  I  ericr- .  PPB.S 
I  n  a  il  I'  q  o  a  (  i  e  ■  .  .  l',  o  n  g  i  e  s  .s  i  o  n  a  t 

1  ir  r  o;.i.iiia;'.e.M.enI  will  not  go  awa\  hv 
I  !■;;  o\  1  ii;',  the  military  from  the 
pioci'ss.  1  do  tn-lieve  that  inv 

ieco  .  pod.il  ions  can  biing  an  elreieiit 
ot  s  t  ,|ti  I  I  1 1  V  to  t  he  cur  r  ent 
env  1  I  orir  enf  th.it  i  -  now  larking, 
h  Hi  t  111’  I  ,  I  bell  eve  t  ha  I  t  h  IS 


recommendation  will  result  in 
sizeable  savings  to  the  cost  of 
procuring  weapon  systems. 

The  implementation  of  the  concept  of 
Total  Quality  Management  and  the 
search  for  continuous  process 
improvement,  dictates  a  revolutionary 
change  along  the  lines  indicated 
above.  I  believe  that  the 
acquisition  professionals  that  are 
currently  a  part  of  the  weapons 
acquisition  process,  would  benefit 
from  these  changes  as  would  the  men 
for  whom  the  weapons  are  intended  - 
the  American  soldier. 
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ABSTRACT 

Organuations  that  survive  the  90s 
will  be  those  that  can  adapt.  And  one  key 
to  a  modem  organization's  adaptive 
capacity  is  an  accounting  system  that 
provides  relevant  and  reliable  information. 

Decision  makers  have  been  placed 
in  a  precarious  position  as  pressure  mounts 
to  modernize  accounting  systems 
commensurate  with  the  technology 
available,  while  simultaneously,  a  justified 
fear  prevents  managers  from  experimenting 
with  accounting  methods  or  procures  that 
may  place  their  cherished  and  hard-earned 
accounting  credibility  in  jeopardy. 

With  this  challenging  business 
environment  stage  set,  tlus  paper  will 
describe  and  explore  specific  issues  and 
concerns  associated  with  implementing 
innovative  cost  management  approaches 
from  both  the  government's  and  defense 
contractor's  perspective.  A  recommended 
potential  course  of  action  will  attempt  to 
guide  decision-makers  through  the  realities 
of  doing  business  in  the  defense  world  in 
the  90s. 

From  the  government's  perspective, 
how  could  anything  that  has  so  many 
people  thinking  of  ways  to  improve 
reporting  and  accountability  be  bad?  But 
there  are  a  host  of  important  issues  that 
must  be  addressed.  First  and  foremost, 
considering  other  accounting  practices  and 
procedures,  ABC  is  relatively  immature. 
Second,  any  change  of  this  type  would 
surely  require  a  lengthy  transition  period. 
Third,  ABC  implementation  is  expensive. 
Fourth,  how  are  ongoing  projects  to  be 
handled?  Finally,  there  is  a  litany  of 
regulations,  laws  and  conventions  that  need 
to  be  addressed. 


From  the  contractor's  perspective, 
there  are  many  issues  to  consider.  First,  it 
is  not  a  wise  investment  at  this  time. 
Second,  most  implementation  issues  are 
unresolved.  Third,  who  really  wants  ABC? 
Finally,  there  are  other  complications  that 
need  to  be  addressed:  loss  of  cost 
estimating  capabilities,  potential  problems 
with  forward  pricing  rates,  cost  proposals, 
negotiations,  cost  performance  data,  and 
potential  "No-Fee"  or  "disallowable"  labels 
on  "non- value  added  activities."  Finally, 
there  is  no  penalty  for  not  adopting  ABC. 

We  feel  if  several  conditions  exist, 
ABC  implementation  will  be  feasible  for 
our  defense  industry  including:  ABC  must 
be  validated  and  accepted  in  other  Helds, 
our  anachronistic  CAS  system  must  be 
modernized  and  the  current  turbulence  and 
uncertainty  in  our  defense  industry  must 
stabilize. 

With  these  conditions  met,  several 
steps  could  be  taken  to  enhance  successful 
ABC  implementation.  First,  ABC  must  be 
encouraged/supported  by  customers  and 
suppliers  alike.  Second,  all  government 
agencies  must  be  fully  aware  of  all  ABC 
activities.  Second,  ke^  it  simple.  Third, 
select  a  pilot  program  and  finally,  be 
extremely  sensitive  to  labelling  non-value 
added  work. 

Clearly  there  is  a  strong  argument 
that  in  our  current  acquisition  environment, 
some  if  not  all  of  these  conditions  and  steps 
could  not  realistically  be  accomplish^. 
For  that  reason  we  feel  that  implementing 
ABC  to  improve  cost  information  might  be 
exactly  the  right  thing  to  do,  but  now  might 
be  exactly  the  wrong  time  to  do  it. 
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Organizations  that  survive  the  90s 
will  be  those  that  can  adapt.  And  one  key 
to  a  modem  organization's  adaptive 
capacity  is  an  accounting  system  that 
provides  relevant  and  reliable  information. 
In  response  to  this  need,  the  accounting 
community  has  developed  what  is  popularly 
referred  to  as  cost  management. 

While  many  authors  depict  cost 
management  as  a  discipline  that  is  finally 
coming  of  age,  others  suggest  that  it  is 
actually  only  in  its  infancy.  Examples 
supporting  both  of  these  positions  are 
vividly  illustrated  when  examining  the 
history  of  cost  management  for 
corporations  that  deal  primarily  with  the 
Department  of  Defense  (DoD).  In  many 
respwts,  our  defense  industry  is  the  most 
sophisticated  and  innovative  sector  in  the 
world.  In  other  respects,  specifically 
dealing  with  pioneering  accounting 
practices  and  procedures,  the  defense 
industry  is  bogged  down  by  laws, 
regulations  and  other  realities  of  today’s 
marketplace. 

Attaining  the  desired  level  of  cost 
management  efficiency  and  effectiveness  is 
a  difficult  and  costly  proposition  for  many 
reasons.  The  unprecedented  capabilities 
ushered  in  by  the  "Information  Age", 
coupled  with  the  current  dynamic  world 
situation  and  a  larger  than  anticipated 
reduction  in  the  need  for  new  major 
weapon  systems,  have  created  a  classic 
"Catch-22"  for  defense  firms.  Decision 
makers  have  been  placed  in  a  precarious 
position  as  oressure  mounts  to  modernize 
accounting  systems  commensurate  with  the 
technology  available,  while  simultaneously, 
a  justified  fear  prevents  managers  from 
experimenting  with  accounting  methods  or 


procedures  that  may  place  their  cherished 
and  hard-earned  accounting  credibility  in 
jeopardy. 

All  of  this  is  happening  at  a  time 
when  investment  capital  for  improvement 
efforts  of  the  dimensions  required  is 
basically  nonexistent.  To  make  matters 
worse,  the  outlook  for  future  business 
appears  be  out-pacing  analysts'  worst  case 
pr^ictions.  There  seems  to  be  no  reward 
for  risk-taking,  but  negative  incentive  for 
inaction. 

This  dilemma  is  further  complicated 
as  other  state-of-the-art  management 
initiatives  are  embraced  and  discarded  in 
succession.  Today's  decision-makers  find 
themselves  having  to  exercise  extreme 
caution  when  advocating  improvement 
efforts  as  they  may  see  their  favorite 
"management  fad"  vanish  -  along  with  their 
personal  credibility. 

A  final  complication  is  the  mountain 
of  laws  and  regulations  imposed  upon 
defense  contractors  by  the  government. 
While  these  actions  were  taken  mainly  in 
response  to  a  public  outciy  for  increased 
accountability  of  government  spending,  the 
end  result  is  a  heavily  regulated  business 
environment  that  is  generally  change 
adverse. 

With  this  challenging  business 
environment  stage  set,  this  paper  will 
describe  and  explore  specific  issues  and 
concerns  associated  with  implementing 
innovative  cost  management  approaches 
from  both  the  government's  and  defense 
contractor's  perspective.  A  recommended 
potential  course  of  action  will  attempt  to 
guide  decision-makers  through  the  realities 
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of  doing  business  in  the  defense  world  in 


the  90s. 


ACTIVITY-BASED  COSTING 


A  sampling  of  the  popular  business 
press  and  professional  accounting  literature 
would  lead  one  to  believe  the  accounting 
community  has  firmly  embraced  the  fact 
that  revolutionary  change  is  in  motion,  and 
the  transportation  mode  of  choice  is 
Activity-Based  Costing  (ABC);  further,  the 
ABC  train  has  left  the  station  and  continues 
to  pick  up  passengers  at  each  stop.  ABC 
(also  referred  to  in  a  broader  concept  as 
activity-based  management)  has  as  a 
primary  purpose,  to  provide  information 
for  decision  makers'  internal  use.  ABC 
also  permits  the  decision  maker  to 
concentrate  on  the  management  of 
resources  instead  of  external  or  internal 
requirements. 

While  this  is  exactly  what  traditional 
cost  and  management  accounting  provided 
in  the  past,  they  seem  to  have  experienced 
a  savage  barrage  of  criticisms  from 
academics  and  practitioners  alike. 
Conventional  cost  system  bashing  is  easy, 
fun  and  makes  you  look  smart!  Also, 
advocating  traditional  "absorption"  type 
cost  systems  grows  increasingly  difficult 
when  such  systems  consistently  fail  to 
provide  relevant  and  timely  information, 
especially  in  an  era  of  scarce  resources 
associated  with  the  modem  workplace. 

More  specifically,  ABC  is  a  method 
of  measuring  cost  and  activity  performance 
associated  with  production  or  services.  The 
idea  behind  ABC  is  that  products  or 
services  require  activities  and  activities 
consume  resources.  A  manager  will 
essentially  assign  costs  to  cost  objectives 
based  on  their  use  of  a  specific  activity. 
The  most  obvious  benefit  is  that  instead  of 
merely  information  on  how  costs  are 
consumed,  the  work  being  performed  is 
highlighted. 

ABC  and  the  apparent  simultaneous 
emergence  of  Total  Quality  Management 
(TQM)  on  the  contemporary  business  scene 
in  the  mid-1980s  gave  managers  a 


staggering  one-two  punch  just  as  their 
backs  were  figuratively  against  the  ropes. 

TQM,  with  an  associated 
philosophy  of  customer  satisfaction  and  a 
focus  on  process  improvement,  highlighted 
that  activities  can  be  managed,  unlike  costs 
that  are  basically  collect^  and  reported 
after  the  fact.  By  eliminating  non-value 
added  activities  (work)  associated  with  the 
processes  (primary  functions)  of  the 
organization,  costs  could  not  only  be 
accounted  for,  collected  and  reported  in  a 
more  timely  manner,  they  would  prove  to 
be  more  representative  of  what  actually 
occurred  on  the  production  line  or  at  the 
service  window.  Even  more  appealing  was 
the  fact  that  these  activities  could  be 
systematically  and  judiciously  "reduced"  by 
managing  the  activities  that  actually 
consume  the  resources  as  opposed  to  the 
traditional  orientation  of  products  absorbing 
costs.  This  entire  management  process  can 
be  oriented  toward  continuous  improvement 
and  before  long,  TQM  is  virtually  in 
perfect  harmony  with  ABC  efforts. 

There  are  many  that  would  probably 
argue  that  ABC  evolved  out  of  a  need  to 
accurately  address  the  seemingly  out-of¬ 
control  and  ever-inceasing  overhead  (or 
common  cost)  accounts.  Although  not  the 
sole  culprit,  automation  has  decreased  the 
labor  content  of  certain  functions  and 
significantly  increased  the  general  overhead 
levels  across  the  board.  This  phenomenon 
made  ABC,  with  its  focus  on  allocation  of 
overhead  particularly  attractive.  Instead  of 
allocating  all  of  this  overhead  by  some 
volume  related  metric  (direct  labor  hours 
etc),  overhead  is  decomposed  and  allocated 
by  more  appropriate  "cost  drivers." 
Traditional  cost  elements  (direct 
labor/material)  would  still  be  used  when 
appropriate,  but  now  would  be  accumulated 
by  activity  and  allocated  by  appropriate 
cost  driver. 
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The  end  result  of  improvements  to 
costing,  such  as  ABC,  is  a  potential 


increase  to  the  bottom-line  (pnjfits).  This 
alone  might  be  what  caused  eyebrows  to 
rise  in  corporate  boardrooms  across  the 
countiy  as  controllers  began  advocating 
ABC  implementation.  But  what  about  the 

ABC  IMPLEMENTATION  ISSUES  FROM 

From  the  government's  perspective, 
how  could  anything  that  has  so  many 
people  thinking  of  ways  to  improve 
reporting  and  accountability  be  bad? 
Implementing  ABC  is  of  such  interest  to 
everyone  because  it  directly  affects  such 
fundamental  activities  as  contract  payment, 
future  contract  negotiation,  cost 
performance  and  cost  estimating  and 
analysis.  Further,  all  of  these  have 
potential  impact  on  the  bottom  line  -  profit. 

Simply  put,  profits  are  a  function  of 
cost.  And  profits  are  where  DoD  agencies 
exercise  the  lion's  share  of  their 
negotiations.  Consequently,  both  parties 
have  ample  incentive  to  possess  accurate, 
timely  and  relevant  cost  data  and  then 
reduce  those  costs  once  identified.  The 
recent  emphasis  by  many  authors  on  cost 
management  extends  beyond  cost  control  or 
cost  accounting  to  touch  at  the  complex 
relationships  between  cost  and  price.  This 
could  be  the  primary  reason  ABC  has 
received  so  much  positive  press.  It  creates 
a  virtual  line-of-sight  view  of  how  the  mix 
of  an  organization's  diverse  activities  and 
processes  contribute  to  the  bottom  line,  and 
provides  a  credible  audit-trail  as  well.  This 
is  particularly  tantalizing  to  decision¬ 
makers  who  have  already  accepted  (in 
principle)  or  have  already  invested  time 
and/or  money  into  improvement  efforts 
such  as  TQM. 

With  all  of  ABC's  attributes,  it  is 
hard  to  understand  why  the  government 
might  balk  at  this  apparent  "win-win" 
situation.  But  there  are  a  host  of  important 
issues  that  must  be  addressed. 

First  and  foremost,  considering 
other  accounting  practices  and  procedures, 
ABC  is  relatively  immature.  The  DoD 
cannot  afford  to  experiment  on  a  large  scale 
with  a  basically  "unproven"  concept. 


public  sector?  Can  the  government,  and 
specifically  the  DoD  reap  the  same  benefits 
that  ABC  seems  to  be  providing  the  private 
sector? 


THE  GOVERNMENT'S  PERSPECTIVE 


Second,  any  change  of  the 
magnitude  associated  with  implementing  a 
new  cost  system,  would  surely  require  a 
lengthy  transition  period.  During  this  time 
span,  there  might  be  a  requirement  for 
"two  sets  of  books",  if  not  more. 
Accountability  becomes  difficult  and 
credibility  will  always  be  on  the  line. 

Third,  ABC  implementation  is 
expensive.  From  early  feasibility  studies, 
to  the  actual  software,  to  the  huge 
investment  of  manpower,  ABC  is  an 
expensive  proposition.  It  is  also  critical  to 
note  that  there  would  be  a  huge  opportunity 
cost  of  a  firm's  top  financial  staff  working 
on  the  ABC  transition. 

Fourth,  how  are  ongoing  projects  to 
be  handled?  This  is  especially  pertinent  to 
the  large,  multi-year  efforts  that  are  often 
the  lifeblood  of  many  defense  contractors. 
Do  we  change  the  accountability  and 
performance  measurement  rules  of  the 
game  halfway  through  the  life  of  a 
program,  or  are  some  programs 
grandfathered? 

Finally,  there  is  a  litany  of 
regulations,  laws  and  conventions  that  need 
to  be  addressed  and  resolved  before  ABC 
can  successfully  be  implemented.  Of  most 
urgency  are  government  mandated  Cost 
Accounting  Standards  (CAS),  and  the 
potential  flagrant  violations  ABC  could 
effect.  The  key  to  successfully  dealing 
with  CAS  is  to  have  a  cost  system  that 
records  contract  costs  and  provides  relevant 
and  useful  information.  Even  though  the 
use  of  ABC  to  assign  corporate  costs  to 
divisions  or  business  units  particularly  suits 
defense  contractors,  implementing  ABC 
systems  would  require  a  comprehensive 
look  at  the  twenty  cost  standards  issued  by 
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the  Cost  Accounting  Standard  Board 
(CASE). 

Of  particular  note  would  be  CAS 
402,  "Consistency  in  Allocating  Cost 
Incurred  for  the  Same  Purpose",  and  CAS 
406,  "Cost  Accounting  Period."  All 
organizations  concerned  would  have  to 
demonstrate  great  flexibility  as  the  unique 
nature  of  each  organization  and  of  each  cost 
are  addressed.  Finally,  CAS  410/418/403, 
generally  "the  Allocation  of  Indirect 
Costs",  would  be,  to  borrow  a  phrase,  "the 
difference  that  makes  a  difference."  Even 
though  this  area  would  experience 
transition  pains,  this  is  where  ABC  will 
either  shine  or  disappear.  But  in  general, 
the  use  of  a  "causal/beneficial  base"  is 


preferred  to  the  use  of  a  general  or  residual 
base  that  cannot  be  charged  directly.  ABC, 
with  its  cost  driver  orientation,  appears  to 
satisfy  CAS.  However,  ABC 

implementation  would  probably  require 
monumental  change/flexibility  by  the 
CASE. 

In  our  current  era  of  shrinking 
defense  budgets,  ABC  systems  that  are 
reliable,  efficient  and  accurate  would  prove 
to  be  of  immeasurable  value  to  all  parties 
concerned.  The  DoD,  it  would  seem,  has 
much  to  gain  through  the  adoption  of  ABC. 
The  question  remains  as  to  whether  the 
benefits  outweigh  the  costs. 


ABC  IMPLEMENTATION  ISSUES  FROM  THE  CONTRACTOR'S  PERSPECTIVE 


The  bottom  line  is:  if  ABC  is  such 
a  great  idea,  why  are  aerospace  contractors 
not  knocking  themselves  out  trying  to  be 
the  first  to  fully  implement  it?  Are  they  so 
rooted  in  antique  accounting  practices  and 
procedures  that  they  fail  to  recognize  the 
benefits  of  ABC?  Maybe,  there  is  another 
side  to  the  ABC  story. 

Assume  that  you  are  the  CEO  of  a 
moderate  size  aerospace  firm  of  5,000  to 
7,000  employees  with  sales  of  one  billion 
dollars  last  year,  (of  which  90%  were  DoD 
agencies).  You  have  recently  presided  over 
down-sizing  10-15%  of  your  employees, 
sales  forecasts  are  falling  through  the  floor, 
burden  rates  are  rising  exponentially  and 
your  work  force  is  partially  paralyzed  by 
"layoff  anxiety." 

Not  a  pretty  picture,  but  you  are 
convinced  that  with  judicious  financial 
constraint,  product  area  consolidation,  and 
aggressive  bidding  for  the  limited 
opportunities  that  do  exist,  your  firm  can 
ride  out  this  cycle.  But  for  now,  the 
shriveling  DoD  budget  calls  for  prudence 
and  budgetary  vigilance  on  all  fronts. 

It  is  in  this  environment  that  your 
controller  convenes  a  high  level  review  to 
present  to  you  his  recommendation  to 
pursue  something  called  ABC.  The  pitch 


crisply  describes  the  theoretical  power  of 
ABC  to  better  define  costs,  illuminate  the 
vague  world  of  indirect  costs,  and  highlight 
non-value  added  activities.  It  is  also  quite 
evident  that  your  controller,  as  an 
accounting  and  financial  professional,  is 
enthusiastic  about  the  concept  of 
implementing  ABC  in  your  company. 
Although  you  clearly  see  the  value  of  ABC, 
as  CE(5  you  have  to  look  at  each  business 
decision  from  an  investment  jjerspective; 
specifically,  what  is  the  projected  Return 
on  Investment  (ROI)  of  ABC,  and 
simultaneously  what  is  the  penalty  or  cost 
of  not  implementing  ABC  at  this  time? 

After  the  presentation  and  extensive 
dialogue,  you  draw  some  conclusions 
regarding  ABC.  First,  it  is  not  a  wise 
investment  at  this  time.  Implementing 
ABC  involves  tying  up  some  of  your  top 
people  for  3-5  years  researching  cost 
drivers  and  setting  up  various  cost  pools. 
Total  implementation  cost  estimates  range 
from  $15-$40  million  precious  overhead 
dollars  and  the  benefits,  though  attractive, 
are  still  very  much  theoretical.  From  an 
ROI  perspective  in  uncertain  times,  it  just 
does  not  add  up. 
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Second,  the  implementation  issues 
are  formidable  and  unresolved.  Transition 
ABC  requires  a  new  government 


disclosure  statement,  cost  impacts,  massive 
education/training  programs,  new 
estimating  methodologies,  and  an  increase 
in  administrative  cost  to  manage  a 
significantly  more  complex  accounting 
information  system.  Additionally,  no  one 
has  yet  figured  out  how  to  "cut  over"  to 
ABC.  Do  you  grandfather  existing 
programs  under  the  old  accounting 
structure  (so  as  not  to  upset  your  current 
customers  with  cost  impacts  and  new 
accounting  complications)  while  applying 
ABC  to  new  programs?  But  treating  the 
same  costs  differently  on  separate  contracts 
is  a  violation  of  CAS.  Or  do  you  switch  all 
programs  to  ABC  as  of  some  date  and  incur 
the  possible  wrath  of  valued  customers 
who,  prior  to  implementation  of  ABC,  felt 
comfortable  with  cost  behavior  on  their 
programs  and  had  an  Estimate  at 
Completion  (EAC)  they  could  track? 
Unless  you  are  starting  out  with  ABC  in  a 
new  facility  with  new  contracts,  the 
transition  would  appear  to  be  a  lose-lose 
proposition. 

Third,  who  wants  ABC  anyway? 
ABC  is  highly  desirable  for  its  apparent 
ability  to  provide  sorely  needed  insight  into 
cost  and  profit  drivers.  But  when  was  the 
last  time  your  customer  asked  you  how 
your  company  was  doing  with  ABC 
implementation?  The  fact  is  that  not  only 
are  contractors  feeling  zero  pressure  to 
migrate  out  of  current  accounting  systems 
to  ABC,  a  formidable  implementation  task 
would  be  the  education/training  (selling)  of 

WHAT  DOES  THIS  MEAN  FOR 

All  parties  involved  with  major 
weapon  systems  acquisitions  seem  to  agree 
that  ABC  has  merit.  The  DoD  and  the 
contractois  who  have  stepped  up  to  ABC 
appear  to  concur  on  the  fundamental 
benefits  of  an  innovative  accounting  system 
that  might  clear  the  fog  surrounding 
overhead  expenses  and  thus  produce  more 
meaningful  and  accurate  estimates  of 
product  costs.  The  rub  is  associated  with 
implementation;  despite  the  benefits,  the 
realities  of  corporate  operations  in  the 
defense  enclave  demand  that  a  host  of 
issues  be  resolved,  or  at  least  addressed  in 


customers  and  customer  representatives  of 
the  merits  of  ABC. 

Fourth,  there  are  a  few  "Honorable 
Mention"  complications,  including: 

A)  Loss  of  cost  estimating 
(heuristic  and  parametric)  relationships 
developed  over  many  years. 

B)  Potential  increase  (by  a 
magnitude)  of  complexity  of  forward 
pricing  rate  packages,  cost  proposals, 
negotiations,  and  cost  performance 
data/ reports. 

C)  Potential  government  imposition 
of  "No-Fee"  or  "disallowable  cost"  on 
"non-value  added  activities"  or  on  "idle 
capacities. " 

Finally,  there  is  no  immediate, 
significant  penalty  of  not  adopting  ABC. 
In  short,  without  significant  help  and 
regulatory  relaxation  from  agencies  trained 
and  incentivized  to  do  just  the  opposite,  it 
would  appear  that  you  can’t  get  there  from 
here. 

No  external  forces,  government  or 
competitors,  are  driving  the  expenditure  of 
huge  amounts  of  scarce  capital.  And 
considering  that  contractor  accounting 
systems  are  working  well,  it  would  seem 
prudent  to  ride  out  the  storm. 


AND  SYSTEM  ACQUISITIONS? 

open  forum  at  the  highest  levels,  before 
ABC  doctrine  and  practices  can  be 
realistically  embraced.  We  feel  if  several 
conditions  exist,  ABC  implementation  will 
be  feasible  for  our  defense  industry: 

1)  ABC  r»iust  be  validated  and  accepted  in 
other  fields,  disciplines  and  industries.  Its 
theoretical  tenets  must  be  proven  more 
substantially  in  order  to  gain  better 
acceptance.  Unfortunately,  we  recognize 
that  this  is  usually  a  function  of  time;  yet 
another  of  our  scarce  resources. 
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2)  Certainly  our  anachronistic  CAS  system 
must  be  modernized  in  order  to  deal  with 
the  advent  of  ABC.  Current  standards 
discourage  realistic  implementation,  and 
further,  justifiably  fail  to  encourage 
innovation  and  improvement. 

3)  The  current  turbulence  and  uncertainty 
in  our  defense  industry  must  stabilize.  At 
present,  the  resources  do  not  seem  to  exist 
to  attempt  ABC  implementation,  and 
investment  horizons  are  short. 

With  these  conditions  met,  several 
steps  could  be  taken  to  enhance  successful 
ABC  implementation: 

1)  ABC  must  be  encouraged/supported  by 
customers  and  suppliers  alike.  ABC 
education  is  a  neccesity,  and  upper 
management  champions  must  be  developed. 
Tough  job,  especially  as  other  management 
initiatives  have  gobbled  up  all  of  the 
bosses'  attention.  Also,  it  is  imperative 
that  all  government  oversight  agencies 
(DC AS,  DCAA,  etc.)  be  fully  aware  of  all 
ABC  implementation  activities. 

2)  Keep  it  simple.  Do  not  try  to  "reinvent 
the  wheel."  Research  other  defense  firm's 
efforts  as  you  would  benchmark  quality 
efforts.  Pick  one  product  area  with  a  clean 
cross-functional  support  structure.  This 
will  facilitate  recognizable  cost  driver 
identification. 

3)  Select  a  small  autonomous  site  for  a 
pilot  program.  Emphasize  product 
"families"  and  try  to  achieve  an  early 
success  while  addressing  all  required 
disclosure  implications. 

4)  Be  extremely  sensitive  to  labelling 
value/non- value  added  work.  More  than 
just  feelings  might  be  damaged  as  processes 
and  primary  activities  are  defined. 

While  meeting  these  conditions  and 
steps  is  not  a  recipe  for  guaranteed  success, 
they  will  help  organizations  address  ABC 
implementation.  Clearly  there  is  a  strong 
argument  that  in  our  current  acquisition 
environment,  some  if  not  all  of  these 
conditions  and  steps  could  not  realistically 


be  accomplished.  For  that  reason  we  feel 
that  implementing  ABC  to  improve  cost 
information  might  be  exactly  the  right  thing 
to  do,  but  now  might  be  exactly  the  wrong 
time  to  do  it. 


"THE  IDEAS  AND  OPINIONS 
EXPRESSED  HEREIN  ARE  THOSE  OF 
THE  AUTHORS  AND  DO  NOT 
NECESSARILY  REFLECT  THE  VIEWS 
OF  THE  UNITED  STATES  AIR  FORCE 
OR  ANY  OTHER  AGENCY  OF  THE 
DEPARTMENT  OF  DEFENSE.  THIS 
ALSO  HOLDS  TRUE  FOR  MARTIN 
MARIETTA  CORPORATION." 

This  project  was  funded  in  part  by  the 
Defense  Systems  Management  College,  Ft 
Belvoir,  VA. 
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IMPLEMENTATION  OF  THE  INTEGRATED  MANAGEMENT  SYSTEM 
IN  THE  CENTRAL  TACTICAL  PROCESSING  PROGRAM  (CTPP) 

An  Experimental  Request  for  Proposal 

Beryl  A.  Harman,  Defense  Systems  Management  College 


ABSTRACT 

The  Central  Tactical  Processing  Program 
CTPP  is  a  $25  million,  one  of  a  kind, 
experimental  ground  station  in  support  of 
the  Defense  Support  Program  (DSP), 
funded  with  research  and  development 
funds  under  the  Defense  Strategic  Initiatives 
Program  (SDI)  This  critical  experiment 
received  authority  to  proceed  on  6  Mar  92, 
and  was  award^  through  formal  source 
selection  procedures  under  Air  Force 
Regulation  70-30,  without  discussions,  on  3 
Aug  92.  Since  this  program  was  on  an 
accelerated  schedule,  it  was  decided  to 
utilize  acquisition  streamlining  techniques 
and  to  include  the  Integrated  Management 
System  (IMS)  concept  developed  for  the  F- 
22  aircraft  at  the  Aeronautical  Systems 
Center  at  Wright  Patterson  Air  Force  Base. 
In  addition  it  was  decided  to  utilize  an 
Integrated  Product  Team  OPT)  to 
implement  the  acquisition  process.  This 
paper  describes  the  process  that  was 
implemented;  the  iiioovations  that  were 
us^  in  both  the  Request  For  Proposal 
(RFP)  and  contract;  and  the  results/lessons 
learned  in  implementing  the  process. 


INTRODUCTION  -  THE  PROCESS 
DEFINED 

The  Central  Tactical  Processing  Program 
(CTPP)  received  authority  to  proceed  on  6 
Mar  92  and  was  contractually  awarded  less 
than  six  months  later.  The  sy.stem  was 
purchased  with  three  major  events  in  mind. 
The  purchase  of  an  initial  usable  system 
within  a  very  short  time  frame;  purchase  of 


a  Limited  Operational  Capability  (LOC) 
within  eighteen  months  of  award;  and 
growth  from  the  LOC  to  an  Initial 
Operational  Capability  (IOC).  This  was  to 
be  defined  during  the  period  of 
performance  of  the  contract  by  engineering 
analysis,  (dependent  on  available  funding), 
and  coordination  with  Air  Force  Space 
Command. 

It  was  decided  to  purchase  the  system 
using  full  and  open  competition,  pursuant 
to  Air  Force  Regulation  70-30,  Streamlined 
Source  Selection,  under  the  umbrella  of  a 
Cost  Plus  Award  Fee  (CPAF)  contractual 
arrangement.  Due  to  the  limited  budget 
available  ($24.5  million),  it  was  decided  to 
provide  the  budget  in  the  Request  for 
Proposal  (RFP)  and  to  allow  the  successful 
offeror  to  determine  the  extent  of 
compliance  with  the  "A"  type  or  Systems 
Specification  within  available  funding.  This 
specification  was  considered  a  functional 
specification  within  the  dictates  of  systems 
engineering. 

The  "A"  type  specification  was  structured 
with  this  in  mind.  The  LC>C  was 
documented  as  a  set  of  firm  requirements 
that  the  offeror  must  meet  and  additional 
desired  capabilities,  potentially  required  for 
the  IOC,  were  documented  as  goals.  The 
successful  offeror  was  instructed  to  provide 
the  LOC  within  available  funding  and  to 
determine  what,  if  anything,  he  could 
provide  of  the  documented  goals  within  the 
remaining  dollars,  a.ssuming  projected 
unused  funds  remained.  His  only  limitation 
was  to  demonstrate  the  extent  of  his 
sy.stem’s  maturity  during  the  source 
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selection  process  and  to  provide  an 
acceptable  management  plan  that  would 
explain  how  his  system  would  be  expanded, 
without  degradation  of  existing  capability, 
to  meet  the  LOC  requirements.  This 
demonstration  was  designed  to  preclude 
offerors  from  providing  a  paper  product 
with  no  basis  in  fact.  It  was  also  designed 
to  provide  the  Program  Director  with  an 
understanding  of  the  existing  capability  that 
would  be  provided. 

The  decision  was  made  to  include  the 
Integrated  Management  System  (IMS)  as  a 
tool  for  administering  and  managing  the 
resultant  contract.  This  system  first 
developed  for  the  F-22  Aircraft  and 
forming  the  basis  for  the  new  Integrated 
Product  Development  (IPD)  Process  being 
developed  by  the  Air  Force,  requires  that 
the  offeror  provide  with  his  proposal  an 
Integrated  Master  Plan  which  lays  out  the 
major  events  of  the  program, 
accomplishments  to  complete  these  events 
and  realistic  criteria  that  wilt  demonstrate 
these  events  have  been  completed.  In 
addition,  it  requires  that  the  offeror  provide 
an  Integrated  Master  Schedule  that 
identifies  how  and  when  these  events  will 
be  accomplished.  The  initial  submittal  of 
the  Integrated  Master  Schedule  was 
requested  with  the  Request  for  Proposal 
(RFP).  Subsequent  submittals  will 
document  the  progre.ss  of  the  program  and 
highlight  any  slips  or  difficulties  that  the 
re.sultant  awardee  will  experience.  Technical 
Performance  Measures  were  also  requested 
to  be  identified  by  the  offerors  to 
graphically  measure  the  successful  offeror’s 
progress  toward  meeting  identified 
operational  parameters. 

Since  the  decision  was  made  to  incorporate 
the  IMS  as  a  tool,  the  only  mandatory 
delivery  date  provided  to  the  offeror  was 
the  end  date  of  the  contract.  In  addition,  a 
sample  development  schedule  was  provided 
with  the  RFP  to  assist  the  offeror  in 
understanding  Air  Force  expectations. 


The  Program  Director  requested  the  offeror 
to  provide  as  part  of  the  Integrated  Master 
Schedule,  an  initial  implementation  review, 
a  midterm  review  and  a  final  review. 
These  reviews  are  considered  the 
Government  desired  minimum  major 
milestones  of  the  program  and  allow  the 
F*rogram  Director  to  monitor  the  offeror’s 
progress. 

Early  indu.stry  involvement  was  a  tenet  of 
the  acquisition  and  industry  was  involved  at 
the  earliest  possible  time.  Industry 
participation  was  applied  somewhat 
sparingly  however  due  to  the  openness  of 
the  competition  and  the  projected  number 
of  potential  offerors.  An  Electronic 
Bulletin  Board  was  established  and  all 
documentation  was  provided  to 
Midustry  comments  and  feedback  on 
a  reOi  time  basis.  All  questions  were 
promptly  answered  and  an  Industry 

Conference  was  conducted  to  provide 
maximum  information  to  all  potential 

offerors.  At  every  step  industry  was 

informed  that  the  Government  intended  they 
have  accountability  for  the  design  and  that 
the  Government  intended  to  work  with 
them  as  a  team  in  developing  the  program. 
This  tenet  is  al.so  being  adhered  to  today. 

The  Request  for  Proposal  Process 

The  RFP  was  developed  with  .streamlining 
techniques  in  mind  utilizing  a  team 
concept.  The  idea  was  to  produce  a 

quality,  clear,  concise  document  that  could 
be  easily  understood.  The  objective  was 
to  obtain  a  proposal  that  could  be  awarded 
without  discussion.  Specific  elements  are 
described  below. 

Teamwork 

The  RFP  was  developed  by  a  team  concept. 
Representatives  of  all  major  functional 
areas  were  assigned  to  work  with  the  team 
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including  a  program  management  staff 
representative.  Every  member  met  and 
participated  as  part  of  the  team  on  a  daily 
basis.  Each  member  had  an  equal  voice  in 
the  proceedings  with  the  team  leader,  the 
Program  Director,  acting  as  the  point  of 
final  authority.  The  team  leader  actively 
sought  and  accepted  input  from  all  the 
members  of  the  team  and  often  sought 
confirmation  from  other  members.  The 
initial  or  core  team,  which  developed  the 
REP,  remained  intact  throughout  the  RFP 
process  into  contract  award  with  only  one 
exception.  After  RFP  release  the  Deputy 
to  the  team  leader  was  released  and  the 
Program  Director  assumed  those  duties. 
The  balance  of  this  team  remained  together 
after  contract  award  and  is  currendy 
administering  the  contract. 

In  order  to  stay  on  schedule  and  assure  that 
all  the  team  members  were  actively 
participating,  a  template  was  developed 
using  Microsoft  Project  that  documented 
every  task  that  needed  to  be  performed  and 
the  required  completion  dates.  Each  task 
was  then  assigned  an  implementation 
officer  from  within  the  team.  Each  day 
this  schedule  was  reviewed  to  determine  if 
any  problems  had  arisen  to  cause  schedule 
delays.  If  something  had  happened,  a  work 
around  schedule  was  immediately  developed 
or,  if  necessary,  additional  resources 
assigned.  In  this  manner  the  team  leader 
was  in  constant  contact  with  the  activities 
of  the  team  and  problems  were  dealt  with 
in  a  timely  fashion.  Only  those  activities 
that  were  beyond  the  control  of  the  team 
itself  presented  any  real  problem.  The.se 
were  continually  discus.sed  and  elevated  if 
possible  to  resolve.  In  this  manner,  the 
CTPP  team  was  able  to  develop  a  schedule 
for  RFP  release  and  maintain  that  sch  Jule 
with  little  if  no  difficulty. 

RFP  Structure 

TTie  RFP  was  experimental  in  nature  and 
was  structured  using  the  principles  of  the 


Integrated  Management  System.  The 
Integrated  Management  Plan  and  the  initial 
submittal  of  the  Integrated  Master  Schedule 
were  requested  to  be  delivered  with  the 
proposal  on  the  understanding  that  the 
Integrated  Master  Plan  would  become  a 
part  of  the  awarded  contract  and  the 
Integrated  Master  Plan  would  be 
incorporated  by  reference  and  subsequent 
required  submittals  will  document  the 
program.  A  sample  format  for  Plan  and 
Schedule  was  incorporated  in  the  RFP  to 
avoid  confusion. 

Along  with  these  two  documents  the  RFP 
requested  that  the  offerors  provide  an 
explanation  in  narrative  form  on  those 
items  that  would  not  be  specifically 
identified  in  the  Integrated  Master  Plan. 
Particularly  the  processes  that  the  offeror 
would  use  in  completing  the  contract. 
With  the  exception  of  the  "A" 
Specification,  the  Government  provided 
very  few  Military  Standards  or 
Specifications  as  mandatory  requirements 
that  the  offerors  had  to  comply  with.  Even 
those  that  were  provided  could  be  tailored 
by  the  offeror  if  he  could  show  adequate 
cause.  It  was  the  preference  of  the 
Government  that  the  successful  offeror  use 
his  own  processes  in  developing  the  CTPP. 
One  exception  to  this  was  the  cost 
reporting. 

The  offerors  were  required  to  implement 
cost  reporting  using  a  Cost  Performance 
Report  as  part  of  the  contract  deliverables. 
In  addition,  the  offerors  were  required  to 
use  a  "flash  report"  which  required  the 
offeror  to  submit  raw  cost  data  five  days 
after  the  end  of  the  accounting  period. 
This  data  could  then  be  analyzed  on  a  real 
lime  basis  to  determine  if  the  awardee  was 
overrunning  the  contract.  Actual 
performance  under  the  contract  was  to  be 
determined  by  the  use  of  Technical 
Performance  Measures  (TPMs).  These 
measures  could  be  detemiined  after  contract 
award.  Progress  would  then  be  rewarded 


by  the  designated  award  fee  for  a  given 
period. 

The  RFP  structure  was  based  on  one 
numbering  system.  This  system  was 
developed  using  Military  Standard  881  the 
Work  Breakdown  Structure  (WBS)  as  the 
management  base.  A  sample  WBS  stmcture 
was  developed  to  the  fourth  level  and 
provided  for  guidance  in  the  RFP.  The 
offerors  were  allowed  to  change  the  WBS 
structure  below  the  third  level  providing 
that  a  good  explanation  could  be  provided 
and  that  it  fit  their  intended  management 
structure.  There  was  a  conscious  effort  to 
match  the  WBS  elements  with  the  products 
of  the  program  and  avoid  contractor 
reallocation  of  work  effort  and  resultant 
dollars. 

A  sample  Dictionary  was  also  provided  to 
the  offerors  for  information  purposes. 
Great  emphasis  was  placed  on  offeror 
development  of  a  WBS  Dictionary  that 
truly  represented  the  intended  work  package 
deliverables.  The  iritent  of  the 

Government  was  to  use  these  work 
packages  to  trace  the  funding  usage  of  the 
resultant  awardee  and  be  able  to  easily 
match  different  fund  types  with  woric 
expended.  The  Dictionary  had  one  further 
role  and  that  was  to  document  and  explain 
the  work  that  was  to  be  done  on  the 
resultant  contract  The  idea  was  to  have  a 
limited  Statement  of  Work  (SOW),  with  the 
same  numbering  system  as  the  WBS,  which 
would  only  serve  as  an  additional 
clarification  of  the  necessary  work  efforts 
to  be  performed,  but  would  not  stand  as  a 
totally  separate  document.  In  this  way  the 
WBS,  the  Dictionary,  and  the  SOW  were 
tied  together  into  an  integrated  package. 

The  sample  SOW  provided  to  the  offerors, 
incorporated  the  minimum  Government 
requirements  that  the  contractor  had  to 
adhere  to.  The  offerors  were  encouraged  to 
rewrite  those  portions  of  the  SOW  that 
needed  revision  to  adapt  to  the  offerors 


processes  and  proposed  program.  In  this 
manner  the  SOW  was  rewritten  by  the 
offerors  and  had  more  meaning. 

The  "A"  Specification  was  also  integrated 
into  this  numbering  system.  Each  product 
was  clearly  identified  in  terms  of  functional 
requirements  and  specifically  identified  to 
the  projected  WBS  elements.  This 
Specification  was  provided  as  an  attachment 
to  the  RFP  and  resultant  contract.  In  this 
manner,  a  RFP  cross-reference  matrix 
explaining  how  the  different  parts  of  the 
R^  worked  together  became  unnecessary. 
The  RFP  and  resultant  contract  were  fully 
integrated  documents. 

To  enhance  this  structure,  the  data 
requirements  requested  by  the  RFP  were 
kept  to  a  minimum.  Only  fourteen  data 
reporting  requirements  were  requested  in 
the  RFP,  including  the  Integrated  Master 
Schedule,  to  be  delivered  as  part  of  the 
resultant  contract. 

In  addition,  the  Contract  Line  Items 
(CLINS)  were  tied  to  the  WBS  elements. 
Each  CLIN  had  an  identified  WBS, 
Dictionary  and  SOW  requirement.  This 
provided  clear  traceability  of  the  contractual 
line  item  funding  to  the  applicable  WBS 
elements  for  cost  reporting  usage. 

Since  this  acquisition  was  to  be 
accomplished  by  formal  source  selection 
procedures,  the  evaluation  criteria  was  also 
tied  to  the  CLlNs.  Each  CLIN  therefore, 
had  designated  WBS  elements  for  cost 
tracking,  a  Dictionary  that  described  the 
work  packages,  a  SOW  that  clarified  the 
work  effort  and  evaluation  criteria  that  was 
used  in  determining  the  acceptable  design 
for  the  program. 

The  successful  offeror’s  proposal  was  then 
made  an  attachment  to  the  resultant  contract 
so  there  was  no  misunderstanding  of  what 
the  contractor  had  to  offer  and  what  the 
Government  accepted 
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Contractor  Flexibility 


final  RFP  as  issued  held  no  surprises. 
Red  Team  Reviews 


The  design  of  this  acquisition  was  to 
provide  the  offerors  with  maximum 
flexibility.  Therefore,  the  offerors  could 
revise  the  WBS,  the  SOW  and  the 
Standards/Specifications  to  match  not  only 
the  design  they  proposed,  but  also  the  way 
they  intended  to  manage  the  program 
internally.  The  Government  did  not  want 
to  impose  a  contractor  realignment  within 
the  contractor’s  facility  to  match  the 
program  or,  impose  new  processes  on  the 
contractor  for  performing  the  contract.  The 
idea  was  to  utilize  what  the  contractor  had 
in  place  for  maximum  cost  efficiency. 

High  Fidelity  Draft  Request  for  Proposal 
(DRFP) 

At  each  step  of  the  process  the  Government 
desired  to  have  a  high  fidelity  Draft 
Request  for  Proposal  (DRFP).  This  DRFP 
would  allow  Ae  offerors  to  begin  the 
formal  proposal  process  with  an  RFP 
document  that  would  undergo  little  or  no 
change.  In  this  manner,  the  offerors  would 
be  able  to  spend  available  funding  and  time 
on  producing  a  good  proposal  and  not  on 
continuously  changing  the  design  to  meet 
changing  government  requirements. 

Instead  of  issuing  a  low  fidelity  DRFP,  the 
Program  Director  made  maximum  use  of  an 
Electronic  Bulletin  Board  System  to 
incrementally  provide  available 
documentation  for  industry  comment.  Each 
document  that  was  to  become  a  part  of  the 
final  RFP  package  was  put  on  this  bulletin 
board  as  soon  as  it  was  available. 
Comments  were  continuously  solicited  from 
industry  and  periodic  updates  incorporating 
resolution  of  comments  were  provided  for 
further  review.  This  process  provided  real 
time  input  to  the  Program  Director  and 
allowed  industry  to  participate  in 
development  of  the  RFP  documents.  In 
this  manner  a  high  fidelity  DFRP  was 
issued  to  industry  which  assured  that  the 


At  the  time  of  issuance  of  the  DRFP  the 
team  decided  to  hold  red  team  reviews. 
These  were  independent  reviews  of  the 
DRFP  itself  and  were  conducted  in  parallel 
with  other  staff  and  industry  reviews.  There 
were  three  reviews  conducted. 

The  first  review  was  led  by  the  contracting 
officer  and  determined  whether  the  RFP 
was  executable,  fair,  manageable  and 
incorporated  correct  data.  This  was 

considered  necessary  because  the  DRFP 

incorporated  sections  provided  by  different 
individuals  and  disciplines.  It  was 
important  to  ascertain  whether  the  RH*  now 
played  together  as  a  complete  document  or, 
whether  it  was  necessary  to  add  some 
special  provisions  to  clarify  or  explain  the 
government’s  intent.  The  Contracting 

Officer  selected  the  representatives  of  the 
"Contracting  Team"  and  comments  were 
forwarded  to  the  RFP  team  for 

implementation. 

The  second  review  concentrated  on  the 
source  selection  aspects  of  the  RFP.  It  was 
led  by  a  project  officer  and  determined 
whether  the  right  questions  had  been  asked 
to  obtain  the  desir^  information  in  support 
of  the  source  selection;  whether  an 
acceptable  source  selection  plan  had  been 
developed  to  allow  sufficient  time  to 
complete  the  competition;  and  looked  at  the 
relative  weighting  and  priorities  of  the 
source  selection  itself.  Although  specific 
numerical  weights  were  not  assigned  (color 
coding  was  used),  it  was  important  to 
analyze  whether  the  items  were  in  the 
correct  order  of  importance,  whether  the 
right  items  had  been  selected  and  whether 
the  instructions  in  the  RFP  were  sufficient 
enough  to  gamer  the  answers  that  would 
determine  the  best  acceptable  offer.  The 
Project  Officer  selected  the  representatives 
on  the  "Source  Selection  Team"  and 
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comments  were  forwarded  to  the  RFP  team 
for  implementation. 

The  third  review  concentrated  on  the 
perception  of  the  RFP  from  the  standpoint 
of  industry.  In  particular  it  considered 
whether  the  acquisition  could  be  "gamed"; 
i.  e.  could  the  offerors  respond  in  such  a 
way  that  the  government  could 
misunderstand  the  clear  intent  of  the 
proposal  and  thereby  select  the  wrong 
contractor.  It  was  also  concerned  with  lack 
of  information,  lack  of  consistency,  and  the 
relevance  of  the  specifications  and 
standards  that  had  been  included.  It  was 
clearly  understood  by  everyone  that  the 
intent  was  to  award  without  discussion  so 
the  information  provided  needed  to  be 
concise,  clear  and  easily  understandable. 
This  was  of  particular  importance  since  the 
government  chose  to  limit  the  amount  of 
documentation  that  the  offerors  could 
deliver.  The  "Contractor  Team"  was  led  by 
a  staff  representative  and  the  membership 
was  selected  to  provide  an  independent, 
objective  analysis.  Membership  consisted  of 
staff  representatives  who  had  no 
involvement  in  the  acquisition  and 
independent  members  from  the  Aerospace 
Corporation.  The  team  comments  were 
documented  and  forwarded  to  the  RFP  team 
for  implementation 

As  a  result  of  the  red  team  reviews  it  was 
decided  to  forego  a  formai  Solicitation 
Review  Board  (SRB)  process.  The  in  depth 
review  conducted  by  the  smaller  red  teams 
was  considered  more  beneficial  to  the 
process  and  served  the  same  purpose. 

Budget  Provided  in  the  RFP 

The  requirements  of  the  acquisition  did  not 
lend  them.selves  to  an  easily  determined 
cost  estimate.  The  cost  depended  on  the 
extent  of  maturity  of  the  existing  system 
that  would  be  demonstrated.  So  that  the 
necessary  additional  requirements  for  LOC 
were  hard  to  determine.  In  addition,  the 


goals  of  the  acquisition  were  determined  to 
be  major  cost  drivers  that  could  provide  a 
proposal  well  above  the  available  dollars. 
Therefore,  in  order  to  be  fair  and  open 
with  industry  the  government  budget  was 
provided  in  the  RFP. 

It  was  well  recognized  that  including  the 
budget  in  the  RFP  would  make  the 
government’s  job  of  selecting  an  offeror 
more  difficult.  It  became  especially 
important  to  acquire  a  well  laid  out  plan 
and  design  that  could  feasibly  be  provided 
within  the  projected  dollar  amount.  The 
award  fee  plan  criteria  was  therefore 
directed  at  performance  to  the  offeror’s 
management  plan,  and  cost  and  schedule 
performance  within  the  dollars  provided. 
This  left  the  offerors  with  no 
misunderstanding  of  the  government’s  intent 
and  ensured  a  realistic  cost  proposal  would 
be  submitted. 

Major  Building  Blocks  (WBS  and 
Dictionary) 

The  WBS  and  Dictionary  gained 
significant  importance  as  the  process 
progressed.  Not  only  was  the  sample 
structure  established  as  the  basic  building 
blocks  of  the  RFP,  but  it  established  the 
numbering  system  that  was  carried 
throughout  the  "A"  Specification,  the  SOW, 
Sections  L  and  M  of  the  RFP  (Proposal 
Preparation  Instructions  and  Evaluation 
Criteria)  and  the  CLIN  structure.  Each 
second  level  WBS  represented  a  CLIN 
description  that  could  easily  be  adhered  to. 
The  sample  SOW  tasked  against  the 
designated  WBS  and  only  included  the 
necessary  wording  to  clarify  the  intent  of 
the  Dictionary  language.  The  Offerors 
were  not  allowed  to  change  the  second 
level  WBS  structure,  but  could  rearrange  or 
group  lower  level  WBS  items  with  revised 
dictionary  language  provided  the  intent  of 
the  government  was  not  lost  and  there  was 
clear  understanding  of  the  efficiencies  and 
need  involved.  The  sample  WBS  was 
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concurred  in  by  the  financial  staff  prior  to 
RFP  release  and  included  as  an  attachment 
to  the  RFP.  It  was  clearly  understood  that 
the  final  WBS  of  the  program  would  be 
established  during  source  selection  and  that 
no  data  submission  would  be  required  after 
award  of  contract  to  establish  an  acceptable 
WBS  structure. 

Proposal  Structured  in  CLIN  Annexes 

To  further  facilitate  the  source  selection 
process  and  for  ease  of  evaluation,  the 
offerors  were  instructed  to  provide  the 
proposal  in  CLIN  Annexes.  A  sample 
format  was  provided  with  the  RFP  that 
clearly  stated  how  each  Annex  was  to  be 
structured.  Each  annex  not  only  referred  to 
a  CLIN  but  also  to  the  evaluation  criteria 
for  that  CLIN.  In  this  manner  the 
evaluators  would  have  complete 
documentation  that  serviced  each  CLIN 
product  of  the  Contract. 

The  Offerors  were  requested  to  submit  each 
annex  in  the  following  format:  (1)  WBS 
and  Dictionary  definition,  (2)  SOW,  (3) 
references  to  the  "A"  Specification,  (4) 
technical/management  approach  to  the 
program,  (5)  the  Integrated  Management 
Plan  for  that  annex,  (6)  the  Integrated 
Master  Schedule  for  that  annex,  (7)  the 
person  loading  and  required  tooling/facilities 
for  that  annex. 

It  was  clearly  understood  that  it  was  the 
government  intent  to  make  the  proposal  an 
attachment  to  the  contract.  Items  (4)  and 
(7)  however  would  be  added  for 
information  purposes  only.  It  was  not 
considered  appropriate  to  limit  the  offerors 
with  respect  to  design  and  person  loading. 
It  was  fully  recognized  that  the  offeror  had 
to  have  the  flexibilit,;  during  design  to 
make  changes  to  accommodate  the 
functional  requirements  and  to  a.s.sign 
personnel  to  accommodate  that  design.  In 
addition,  since  the  government  would  not 
be  taking  delivery  of  the  "B"  or  "C"  type 


specifications  until  the  end  of  the  program, 
the  offeror  had  full  configuration  control 
over  the  process  until  delivery. 

A  diagram  was  also  included  in  the  RFP 
which  demonstrated  the  difference  between 
the  way  the  offerors  were  used  to  seeing 
RFPs  structured  and  the  new  structure  of 
this  RFP.  Every  opportunity  to  clarify  the 
government’s  intent  was  taken  by  the  RFP 
team. 

Evaluation  Standards  Directly  Relatable 
to  Proposal  Preparation  Instructions. 

Great  care  was  taken  to  relate  the 
evaluation  standards  to  reasonable 
expectations  for  the  program.  The  proposal 
preparation  instructions  only  requested 
information  on  those  critical  items  that 
made  a  difference  in  designing  the  system 
that  would  clearly  distinguish  one  offeror 
from  another  in  terms  of  performance 
capability. 

The  source  selection  team  was  limited  to 
seven  evaluators  and  ten  advisors.  The 
evaluators  were  required  to  read  all  of  the 
proposals  and  assign  a  color  code  to  each 
item  with  inputs  from  the  advisors. 
Therefore  only  discriminators  were 
considered  necessary  inputs  from  the 
offerors  to  make  these  determinations. 

RESULTS  AND  LESSONS  LEARNED 

After  the  source  selection  was  successfully 
accomplished  the  RFP  team  reviewed  the 
process  they  had  undergone  and  considered 
those  things  that  could  have  been  done 
differently  or  better.  A  discussion  of  the 
results  and  lessons  learned  is  included 
below. 

Teamwork 

It  was  extremely  difficult  at  first  for  the 
team  to  work  together  as  a  team.  Each 
member  was  from  a  different  background 
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and  had  different  experience  levels.  To 
work  together  as  a  team  required  that  some 
individuals  undergo  a  cultural  change. 

Each  member  was  from  a  different 
functional  background  and  was 
indoctrinated  to  following  the  precepts  of 
that  functional  discipline.  Working  together 
as  a  group  required  that  some  functional 
requirements  be  modified  and/or  reduced  to 
meet  the  specific  needs  of  the  experimental 
RFP.  It  was  difficult  to  prevent  individual 
members  ft'om  harboring  "hidden  agendas;" 
i.  e.  attempting  to  include  in  the  RFP  their 
own  specific  requirements  attributable  to 
their  functional  discipline.  Once  these 
agendas  became  self  evident  however,  they 
were  easily  dealt  with.  Long  understood 
requirements  were  often  set  aside  to  meet 
the  needs  of  the  acquisition. 

In  order  to  facilitate  good  communication, 
it  was  decided  to  co-locate  the  team  within 
the  same  general  area.  This  became 
difficult  to  arrange  due  to  lack  of  space 
and  matrixing  requirements.  It  was  not 
until  this  process  was  well  underway  that 
this  activity  could  be  accomplished.  Once 
accomplished  however,  the  synergism  of  the 
team  improved  immensely  and 
communication  was  greatly  enhanced. 

Another  problem  that  surfaced  involved  the 
assigning  of  work  effort.  It  became 
necessary,  due  to  the  limited  membership 
of  the  team,  to  make  assignments  that  were 
not  necessarily  based  on  the  expertise  of 
the  individual  or,  within  their  specific 
functional  area.  Therefore,  training  had  to 
be  given  in  order  for  these  individuals  to 
perform.  This  required  specific  one  on  one 
training  and  generated  some  resistance  at 
first.  Some  initial  team  training  might  have 
helped  this  process. 

At  the  conclusion  of  the  process,  there  was 
general  consensus  that  everyone  had  learned 
a  great  deal  about  other  functional  problems 
and  gained  considerably  from  the 
experience.  There  was  a  general  "buy  in"  to 


the  program  as  each  member  became  a 
direct  functioning  member  of  the  team  and 
fully  understood  the  requirements  of  all  the 
members.  There  was  also  a  general 
consensus  that  since  the  requirements  were 
understood  there  was  less  tendency  for 
functional  experts  to  require  the  inclusion 
of  items  that  would  be  expensive  and  of 
little  utilization  to  the  program. 

The  last  positive  comment  concerned  the 
source  selection  process  itself.  Since  each 
member  of  the  team  was  an  evaluator  of 
the  proposal,  had  read  and  understood  the 
entire  RPT*,  and  knew  the  requirements  of 
the  program,  it  was  easier  to  select  the 
acceptable  offer  without  misgivings  and 
without  requiring  discussions. 

Integrated  Management  System 
While  the  team  felt  that  this  area  was 
adequately  covered  in  the  RFP,  some 
difficulty  was  experienced  by  responding 
offerors.  It  appeared  that  the  RFP  failed 
to  provide  a  good  clear  definition  of 
success  criteria  and  the  example  provided 
in  the  proposal  preparations  instructions 
caused  some  misunderstanding.  In  addition, 
the  government  did  not  include  in  the  RFP 
a  clear  description  of  technical  performance 
measures.  This  was  apparent  from  the  fact 
that  no  offeror  demonstrated  a  truly  clear 
understanding  of  what  the  Government 
required  in  this  area. 

An  obvious  solution  to  this  problem  is  of 
course  to  provide  clearer  instructions. 
However,  it  was  felt  that  more  indepth 
discussions  with  potential  offerors  on  a  one 
to  one  basis  would  have  also  helped  the 
situation.  Explicit  government  training 
would  have  greatly  enhanced  the  ability  of 
the  offerors  to  re.spond.  Needless  to  say 
any  future  RFP  structured  along  these  lines 
should  take  this  problem  into  consideration. 
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Work  Breakdown  Structure 

The  respon.ses  received  form  the  offerors  to 


the  WBS  instructions  did  not  change  the 
basic  structure  of  the  RFT*.  However,  the 
changes  that  were  made  lacked  explanation, 
although  clearly  requested  as  part  of  the 
RFP  instructions.  In  addition,  one  minor 
change  actually  drove  the  cost  reporting 
instructions  to  a  lower  level  than  was 
previously  anticipated  or  desired. 

There  is  no  obvious  solution  to  this  kind 
of  problem,  only  alternatives  that  could  be 
considered.  One  potential  alternative  would 
be  to  have  the  offerors  submit  a  WBS 
structure  for  approval  with  the  ORFP  and 
discuss  any  problems  perceived  with  that 
structure  prior  to  formal  RFP  issuance. 
This  would  also  allow  the  financial 
community  to  formally  approve  the 
offeror’s  WBS  structure.  It  could  however, 
have  the  effect  of  limiting  competition  to 
only  those  offerors  with  approved  WBS 
structures  and  could  eliminate  any  offeror 
who  was  late  in  responding  or,  who  had 
difficulty  obtaining  WBS  approval. 
Without  careful  control,  this  could  create  a 
protest  situation.  Another  alternative  would 
be  for  the  Government  to  choose  a  WBS 
structure,  as  a  result  of  industry  inputs, 
with  no  allowance  for  changes.  This  again 
is  not  an  optimal  solution  since  it  restricts 
the  flexibility  of  the  potential  offerors  and 
could  in  fact  favor  a  specific  offeror 
depending  on  which  comments  were 
incorporated.  It  is  hard  to  determine  which 
would  be  the  best  way  to  approach  the 
problem.  It  appears  to  be  a  matter  of 
choice. 

Statement  Of  Work 

The  "Order  of  Precedence"  Provision 
currently  included  in  FAR  and  mandatory 
in  all  government  contracts  caused  some 
confusion  as  to  what  was  more  important 
to  the  Government,  the  SOW  or  the 
specification.  Since  both  the  specification 
and  the  SOW  were  separate  attachments, 
the  RFP  should  have  clearly  stated  that  the 
intent  was  to  have  the  specification  take 


precedence.  Even  though  the  offerors  were 
allowed  to  change  the  SOW  before  it  went 
on  contract,  it  was  not  intended  that  the 
SOW  take  precedence  over  the  Program 
Director’s  "A"  specification.  The  Program 
Director’s  intent  was  to  maintain  and 
control  the  "A"  specification  and  to  manage 
the  program  at  the  performance  level. 
When  one  offeror  made  his  SOW  the 
highest  precedence  document,  this  caused 
some  consternation.  Particularly  since  the 
offeror’s  SOW  could  have  been  inteipreted 
as  changing  some  of  the  performance 
requirements.  The  only  possible  solution  to 
this  kind  of  problem  is  for  the  government 
to  clearly  state  the  order  of  precedence  in 
the  RFP. 

Compliance  Documents 

The  offeror’s  complied  with  the  RFP 
instructions  and  proposed  their  own 
processes  to  meet  the  intent  of  the  military 
standards  or  specifications.  Unfortunately, 
while  these  processes  were  well 
documented,  it  was  difficult  to  determine 
that  in  fact  the  process  proposed  met  the 
intent  of  the  military  standard.  It  then 
became  necessary  to  contact  the.cognbiant 
Administrative  Offices  and  have  tnem 
confirm  that  the  offerors’  processes  did  in 
fact  meet  the  Government’s  intent.  In 
addition,  it  was  found  that  a  subtle 
manipulation  of  the  wording  of  a  paragraph 
could  in  some  instances  completely  change 
the  intent  of  the  requested  standard.  The 
government  had  allowed  the  offerors  the 
flexibility  to  tailor  the  standards,  but  failed 
to  stipulate  that  the  tailoring  had  to  meet 
the  intent  of  the  existing  requirement. 

These  particular  problems  could  have  been 
avoided  with  a  little  forethought.  If  the 
offerors  had  been  asked  to  provide  process 
verification  along  with  their  proposal 
documents,  then  the  government  would 
have  been  able  to  instantly  verify  the 
validity  of  a  process  and  possibly  build  a 
data  base  of  approved  processes.  The  other 
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issue  could  have  been  resolved  if  the 
government  had  clarified  within  the  RFP 
that  any  tailoring  of  the  militaiy  standards 
would  meet  the  intent  of  the  current 
requirements  and  require  the  offeror  to 
clearly  identify  the  tailoring  with 
appropriate  justification.  Hindsight  of 
course  is  always  easy  . 

CLINS  Tied  to  WBS 

The  methodology  of  tying  the  CLINs  to  the 
WBS  had  important  benefits  in  that  it 
provided  clear  traceability  from  the  model 
contract  to  the  SOW  language  and  the 
appropriation.  However,  it  became  clear 
that  when  the  offerors  were  allowed  to 
change  the  WBS  they  also  had  to  change 
the  reference  in  the  CLIN  language. 
Otherwise  that  traceability  is  lost.  Since 
there  were  no  instructions  in  the  RFP  to 
this  effect,  some  offerors  made  the  changes 
and  some  did  not. 

In  addition,  although  the  government 
provided  a  SOW  scope  paragraph,  that 
could  not  be  revised,  as  part  of  the 
minimum  requirements  of  the  SOW,  the 
scope  of  that  document  did  not  tie  directly 
back  to  the  CLIN  structure.  So  that  when 
WBS  changes  affected  traceability  there 
was  a  potential  for  misinterpretation  of  the 
scope  of  the  contract.  The  scope  paragraph 
of  the  SOW  has  direct  significance  in  FAR 
and  ties  directly  into  the  "Changes" 
provision  of  any  resulting  contract.  A  clear 
understanding  of  how  the  CLINs  relate  to 
the  scope  is  essential  to  efficient 
management  of  the  contract.  Providing 
clear  instructions  in  the  RFP  and  resulting 
contract  as  to  how  the  scope  of  the  SOW 
will  be  treated  will  minimize  this  problem. 

Another  feature  of  being  the  WBS  to  the 
CLINs  was  the  necessity  to  create  WBS 
elements  to  provide  meaningful  CLIN 
descriptions  TTie  financial  community  was 
concerned  that  this  activity  would  not  only 
create  unnecessary  cost  reporting,  but  drive 


cost  reporting  to  a  lower  than  desired  level. 
Since  that  was  the  concern,  the  WBS 
elements  that  described  individual  CLINs 
were  raised  to  a  higher  level  for  clear 
traceability  of  appropriations  to  the 
individual  work  packages. 

Contract  Data  Requirements  Lists 

The  government  made  every  attempt  to 
limit  the  amount  of  data  reporting  on  this 
acquisition.  However  the  RFP  was  silent 
on  whether  additional  CDRLs  would  be 
allowed  consistent  with  SOW  changes  or, 
whether  CDRL  tailoring  would  be 
acceptable.  This  became  very  clear  when 
both  of  these  events  occurred  as  part  of 
individual  offeror’s  proposals.  In  fact  one 
offeror  elected  to  tailor  out  the  requirement 
to  provide  "flash  reports"  to  the 
government,  a  basic  requirement  of  the 
RFP.  This  particular  report  was  considered 
a  necessity  by  the  Program  Director  in 
managing  the  contract. 

Since  CDRL  reporting  costs  the  government 
time  and  money,  it  is  important  to  stipulate 
what  the  offerors  can  or,  cannot  do  in 
regard  to  tailoring  or  adding  data 
requirements.  It  is  further  suggested  that 
any  tailoring  be  accompanied  by  full 
identification  and  justification  for  the 
changes  for  ease  of  evaluation. 

Evaluation  Criteria 

The  most  important  thing  that  became 
readily  apparent  was  the  impossibility  of 
having  a  meaningful  separate  management 
area  for  evaluation  purposes.  The 
Integrated  Management  System 
systematically  ties  technical  activities  to  the 
management  of  the  program.  The  processes 
that  support  development  of  the  program 
are  described  and  included  as  part  of  the 
narrative  associated  with  the  Integrated 
Management  Plan.  Therefore,  a  technical 
activity  is  not  fully  described  unless  it  has 
an  acceptable  plan  and  schedule.  These  are 


272 


integrated  products  and  not  severable 
entities.  A  problem  can  exist  however,  if 
sufficient  emphasis  is  not  placed  on 
management  as  an  integral  part  of  the 
evaluation  process.  Management  of  the 
program  gets  lost  in  lieu  of  technical 
requirements  and  the  decision  for  award  is 
based  purely  on  technical  considerations. 
This  being  the  case  the  Program  Director 
was  concerned  that  the  management  of  the 
program  had  adequate  consideration. 

The  offerors,  as  part  of  the  proposal  were 
also  asked  to  document  how  they  would 
organize  their  program  to  support 
development  of  the  system.  The  WBS 
established  a  structure  from  which  the 
offerors  could  develop  Integrated  Product 
Teams  to  manage  the  program.  It  was  the 
intent  of  the  government  to  parallel  this 
structure  in  some  fashion,  for  ease  of 
communication.  The  idea  being  to  stmcture 
a  program  that  best  allowed  the  awardee 
and  the  government  to  work  together  in 
partnership  to  produce  a  viable  system. 
Therefore,  the  management  of  the  program 
was  an  integral  part  of  the  technical 
process. 

The  solution  was  to  evaluate  both  technical 
and  management  as  one  area  of  emphasis 
and  create  separate  items  or  factors  for 
technical  and  management  activities.  The 
CTPP  RFP  leant  itself  to  determining 
management  at  the  factor  level.  This 
worked  really  well  in  determining  the 
acceptable  offeror 

CONCLUSION 

The  CTPP  RFP  process  was  a  valuable 
experience  in  teaming  and  streamlining  the 
R^  process.  The  resultant  contract  was 
awarded  on  3  Aug  92.  The  source 
selection  was  completed  in  record  time  and 
award  was  made  without  discussions  taking 
place.  Everyone  involved  in  the  process 
agreed  that  more  information  and  insight 
had  been  provided  on  this  acquisition  than 


had  ever  been  available  before.  As  a  result 
this  process  established  a  well  documented 
program.  A  few  minor  inconsistencies 
noted  by  the  evaluation  team  were  easily 
conected  after  award  of  the  contract.  The 
same  team  that  created  the  RFP  is  currently 
managing  the  process  and  the  program  is 
making  excellent  progress.  It  is  this  kind 
of  experiment  that  helps  the  government 
understand  what  efficiencies  will  and  will 
not  work  in  a  bureaucratic  environment. 
Only  time  will  tell  whether  in  fact  this 
program  will  be  truly  successful.  It  will  be 
interesting  to  return  to  this  program  one 
year  from  now  and  see  if  the  program  is 
still  on  schedule  and  within  cost. 
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Abstract 

This  research  identified  the 
psychological  types  of  government  and 
private  industry  contract  negotiators,  and 
frequency  of  use  of  negotiation  tactics  and 
strategies.  Analysis  also  shows  statistically 
significant  differences  between  industry 
and  government  negotiators  in  frequency 
of  use  on  20  out  of  33  tactics  and  five  out  of 
eleven  strategies,  and  that  contract 
negotiators  as  a  group  are  statistically 
different  from  the  general  population  in 
terms  of  psychological  type  composition. 

Introduction 

The  negotiation  of  contracts 
awarded  by  the  United  States  Government 
is  an  area  ripe  with  mis-perceptions  on  the 
part  of  the  American  public  as  to  the 
quality  and  qualifications  of  govermnent 
procurement  officials.  Federal  negotiators 
are  viewed  by  the  public  as  being  at  a 
competitive  disadvantage  to  their 
contractor  counterparts  because,  if  for  no 
other  reason,  of  a  lack  of  experience  and 
training.  Indeed,  one  study  indicates  that 
federal  contract  negotiators  are  not  as  well 
educated  or  as  well  trained  in  the 
procurement  field  as  their  industry 
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counterparts  (Mavroules,1991).  A  1986 
study  by  Peterson  reported  that  34%  of  Air 
Force  Logistics  Command  (AFLC)  contract 
negotiators  did  not  possess  a  bachelor’s 
degree  (Peterson,1986:30).  Catlin  and 
Faenza  reported  in  their  1985  study  that 
10%  of  the  contract  negotiators  in  the  Air 
Force  Systems  Command  (AFSC  --  the 
organization  charged  with  the 
responsibility  for  development  and 
acquisition  of  major  weapon  systems  for 
the  Air  Force)  did  not  have  a  bachelor's 
degree  (Catlin  &  Faenza,  1985:38).  Catlin  & 
Faenza  also  reported  that  69%  of  the  AFSC 
negotiators  had  attended  a  workshop  in 
negotiations,  although  the  length  and 
content  of  the  workshop(s)  were  not 
defined  (Catlin  &  Faenza,  1985:39). 

One  means  by  which  government 
negotiators  might  level  the  playing  field 
with  their  better-trained  and  more 
experienced  industry  counterparts,  is  by 
the  understanding  and  proper  utilization  of 
contract  negotiation  tactics  and  strategies. 

The  use  of  negotiation  strategies  and 
tactics  is  the  focus  of  many  books  and 
research  studies  on  the  subject  of 
negotiation  (Karrass;  Nierenberg;  Fisher, 
Ury  &  Patton).  For  the  most  part,  the 


authors  all  agree  that  the  effective  and 
proper  use  of  negotiation  strategies  and 
tactics  is  vital  to  gain  a  satisfactory  outcome 
from  the  negotiation. 

To  establish  a  common  ground  of 
understanding,  the  following  definitions 
are  presented. 

Tactic  (Technique)  is  any 
specific  action,  words  or 
gestures  designed  to  achieve 
both  an  immediate  objective 
(such  as  countering  an  action  by 
the  other  negotiating  party)  and 
the  ultimate  objective  of  a 
particular  strategy.  (Catlin  and 
Faenza,  1985:99) 

Strategy  is  an  organized  plan  or 
approach  to  negotiations  from 
an  overall  perspective  that  may 
be  composed  of  one  or  more 
than  one  tachc.  (Catlin  and 
Faenza,  1985:99) 

One  concern  of  recent  research 
efforts  has  focused  on  personality  type  and 
the  preference  of  contracting  professionals 
for  particular  negotiation  tactics  and/or 
strategies.  Major  Charan  Johnstone  (1986) 
studied  the  relationship  of  psychological 
type  as  measured  by  the  Myers-Briggs 
Type  Indicator  (MBTI)  to  preferred 
negotiation  strategies  and  tactics  of  Air 
Force  negotiators.  The  MBTI  is  a  self- 
reported  survey  instrument  that  indicates  a 
person's  preferences,  and  is  a  validated, 
widely  accepted  instrument  for 
psychological  testing.  While  Johnstone  was 
not  able  to  correlate  the  two  variables,  she 
did  offer  the  possibility  that  the  lack  of 
win-win  choices  on  the  survey 
questionnaire  might  have  led  to  the  lack  of 
correlation  (Johnstone,  1986:118). 


Determining  psychological  type  for 
both  government  and  industry  negotiators, 
and  the  relationship  to  negotiation  tactics 
and  strategies  may  enable  organizations  to 
identify  desirous  traits  for  recruiting  and 
training  contract  negotiators.  The 
significant  problem  is  that  the  information 
available  to  researchers  today  is 
inconclusive  as  to  the  level  of  use  by 
contract  negotiators  of  negotiation  tactics 
and  strategies.  Also,  while  there  have 
been  numerous  studies  addressing 
government  negotiators,  there  has  been 
little  research  investigating  contract 
negotiators  in  the  private  sector.  There  has 
also  been  little  or  no  research  done 
comparing  the  tactics  and  strategies  used 
by  government  and  industry  negotiators. 
Finally,  there  has  been  only  one  study 
comparing  personality  type  to  negotiation 
tactics  and  strategies,  and  that  study  was 
inconclusive.  The  purpose  of  this  research 
was  to  determine  a)  what  negotiation 
tactics  and  strategies  are  used,  and  how 
often;  b)  the  relationship  between 
government  contract  negotiators  and 
industry  negotiators  in  terms  of  what 
tactics  and  strategies  are  used  by  each 
group,  and  how  often;  and  c)  if 
government  contract  negotiators,  as  a 
population,  differ  from  their  industry 
counterparts  in  terms  of  psychological 
composition.  The  research  questions  to  be 
addressed  were:  1)  What  negotiation 
strategies  and  tactics  are  most  frequently 
used  by  contract  negotiators?;  2)  What  is 
the  personality  type  composition  of 
contract  negotiators;  and  3)  Is  there  a 
difference  in  negotiation  strategies  or 
tactics  used  by  government  and  industry 
negotiators?  Two  other  questions  used  in 
the  research  are  not  addressed  in  this 
paper. 
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Negotiation  Tactics  and  Strateeies 

While  the  terms  tactic  and  strategy 
have  their  own  separate  definitions,  in 
reality  it  is  difficult  to  distinguish  whether 
a  certain  action  is  a  tactic  or  part  of  a 
strategy  (Nierenberg  1986:154). 

The  one  common  theme  prevalent  in 
the  literature  concerning  negotiation  is  the 
importance  of  tactics  and  strategies  in 
negotiations.  As  a  result,  each  formal 
writing  usually  contains  a  minimum  of  one 
and  many  times  several  chapters  on  the 
topic  of  negotiation  tactics  and  strategies. 
For  example,  Chester  L.  Karrass  devotes  an 
entire  book,  entitled  Give  and  Take,  to  the 
detailed  explanation  of  the  use  and 
importance  of  two  hundred  negotiation 
tactics  and  strategies.  Karrass  also  stresses 
the  importance  of  both  offensive  and 
defensive  strategies  in  successful 
negotiations  (Karrass,  1974). 

Another  example  of  an  author 
stressing  the  point  that  the  use  of  tactics 
and  strategies  is  vital  in  negotiations  is 
provided  by  a  quotation  from  Negotiating 
To  Win,  by  Peter  Economy, 

Tactics  and  strategies  are  an 
integral  part  of  the  long  history 
of  negotiation.  Whether  you 
choose  to  use  certain 
techniques  or  not,  you  should, 
at  the  very  least,  be  familiar 
with  the  more  prevalent  ones. 

This  way  you  will  be  prepared 
to  counter  their  use  and  better 
defend  your  positions.  When 
you  negotiate  for  a  living,  you 
need  to  have  every  possible  tool 
at  your  disposal  to  use  in  your 
transactions.  (Economy, 

1991:179) 

While  it  is  apparent  that  the 


preponderance  of  the  literature 

emphasizes  the  importance  of  using 

negotiation  tactics  and  strategies,  most 

authors  identify  and  display  a  list  of  tactics 
and  strategies  without  much  guidance  on 
when  or  how  to  effectively  use  strategies 
(Karrass,  Economy  and  others).  One  reason 
for  this  appears  to  be  the  complex  and 
varied  nature  of  each  negotiation.  As  noted 
by  Steele,  Murphy,  and  Russill, 

No  general  rules  can  be  laid 
down  about  tactics.  Each 

negotiation  must  be  considered 
separately  before  you  decide 
which  tactics  are  appropriate. 

It  is  equally  essential  to 
consider  the  persrmalities  end 
approaches  of  the  other  party 
or  parties  to  the  negotiation.  A 
particular  tactic  will  work 
better  on  some  people  than  on 
others.  The  same  tactics  will 
also  work  differently  on  the 
same  person  in  different 
circumstances  or  at  different 
times.  (Steele,  Murphy  & 
Russill,  1989:94) 

Preferred  Negotiation  Tactics  and 
Strategies 

Catlin  and  Faenza  identified  from  a 
list  of  given  alternatives  the  five  moot 
preferred  negotiation  strategies  and  the  top 
ten  tactics  favored  by  278  U.S.  Air  Force 
contract  negotiators  at  four  separate  Air 
Force  Systems  Command  (AFSC)  buying 
divisions  (Catlin  and  Faenza,  1985:vii). 
Due  to  the  exploratory  nature  of  their 
research,  Catlin  and  Faenza  could  not 
explain  why  the  respondents  answered  the 
way  they  did.  Catlin  &  Faenza's  definitions 
of  negoHation  tactics  and  strategies  reflect 
those  of  the  literature  surveyed  (Catlin  and 
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Faenza,  1985:7).  The  U.S.  Air  Force 
negotiators  from  AFSC  selected  the 
following  strategies,  in  rank  order  of 
preference,  from  a  possible  ten  choices: 
Bottom  line;  Statistics  Participation  or 
involvement;  Combination  or  the  big  pot ;  Step- 
by-step 

These  same  negotiators  identified 
their  top  ten  preferences  for  negotiation 
tactics  from  33  possible  choices  as:  Ask  for 
lots  of  data,  Belabor  fair  and  reasonable.  Split 
the  difference  offers.  Allow  face-saving  exits. 
Off-the-record  discussion.  Call  frequent 
caucuses,  Loiv-ball  offers.  Refer  to  your  side's 
generosity.  Escalate  to  opponent's  boss,  Escalate 
to  your  boss  (Catlin  and  Faenza,  1985:41). 

In  1986,  a  second  study  on 
negotiation  tactics  and  strategies  was 
conducted  by  Peterson.  The  difference 
between  the  two  research  studies  was  that 
instead  of  surveying  AFSC  contract 
negotiators  as  Catlin  and  Faenza  had, 
Peterson  surveyed  Air  Force  Logistics 
Command  (AFLC)  contract  negotiators. 
The  ninety-two  responses  from  the  AFLC 
contract  negotiators  were  similar  to  their 
AFSC  counterparts  and  showed  a 
preference  for  the  following  negotiation 
strategies  in  rank  order  as:  Statistics, 
Participation,  Step-by-step,  Bottom  line,  and 
Combination  (Peterson,  1986:43). 

Peterson  asked  the  participants  to 
rank  order  their  five  most  preferred 
negotiation  tactics  from  the  list  of  33 
possible  choices.  The  same  AFLC  contract 
negotiators  identified  their  preference  for 
the  top  five  negotiation  tactics  as  follows: 
ask  for  lots  of  data,  belabor  fair  and  reasonable, 
split  the  difference  offers,  refer  to  your  side's 
generosity,  and  allow  face-saving  exits 
(Peterson,  1986:68). 

Personality  Type  Theory 

In  1923,  the  noted  psychologist  Carl 
G.  Jung  published  the  book  Psychological 


Types.  Within  this  publication,  Jung 
detailed  his  theory  of  personality  types  that 
stated  "much  seemingly  chance  variation  in 
human  behavior  is  not  due  to  chance,  but  is 
in  fact  the  logical  result  of  a  few  basic, 
observable  differences  in  mental 
functioning"  (Myers  and  Myers,1980:l). 
Katherine  Briggs  and  her  daughter  Isabel 
Briggs  Myers,  over  the  course  of  more  than 
forty  years  of  observation  and  research, 
extended  and  expanded  Carl  Jung's  writing 
on  dominant  and  auxiliary  functions  to 
produce  a  systematic  psychological  type 
theoiy  that  integrated  the  primary  and 
secondary  functions. 

The  cornerstone  of  personality  type 
tlieory,  as  expounded  upon  by  Briggs  and 
Myers,  can  be  summed  up  in  the  first 
paragraph  of  Isabel  and  Peter  Myers'  book. 
Gifts  Diffiring: 

It  is  fashionable  to  say  that  the 
individual  is  unique.  Each  is 
the  product  of  his  or  her  own 
heredity  and  environment  and, 
therefore,  is  different  from 
everyone  else.  From  a  practical 
standpoint,  however,  the 
doctrine  of  uniqueness  is  not 
useful  without  an  exhaustive 
case  study  of  every  person  to  be 
educated  or  counseled  or 
understood.  Yet  we  cannot 
safely  assume  that  other 
people's  minds  work  on  the 
same  principle  as  our  own.  All 
too  often,  others  with  whom  we 
come  in  contact  do  not  reason 
as  we  reason,  or  do  not  value 
the  things  we  value,  or  are  not 
interested  in  what  interests  us. 
Seemingly  chance  variation  in 
human  behavior  is  not  due  to 
chance;  it  is  in  fact  the  logical 
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result  of  a  few  basic,  observable 
differences  in  mental 
functioning.  (Myers  and  Myers, 

1980:1) 

The  differences  in  mental 
functioning,  referred  to  above  by  Myers 
and  Myers,  relate  to  the  way  that 
individuals  prefer  to  perceive  and  make 
judgments  when  they  are  given  choices. 
Each  function  is  characterized  by  a 
dichotomous  scale  on  which  each 
individual  has  a  preference  for  choosing 
toward  one  end  or  the  other  when  given  a 
chance  to  choose  between  the  two 
diametrically  opposed  functions.  While 
certain  situations  may  dictate  that  one 
function  be  used  over  the  other,  when 
given  a  choice,  individuals  will  show  a 
propensity  to  exhibit  behavior  that 
consistently  favors  one  end  of  the  scale  over 
the  other  end.  In  fact. 

Each  of  us  develops  a 
preference  early  in  life  and 
sticks  with  it.  And  the  more  we 
practice  those  preferences-- 
intentionally  or 

unintentionally—the  more  we 
rely  on  them  with  confidence 
and  strength.  That  doesn't 
mean  we're  incapable  of  using 
our  non-preferences  from  time 
to  time.  In  fact,  the  more  we 
mature,  the  more  our  non¬ 
preferences  add  richness  and 
dimension  to  our  lives. 
However,  they  never  take  the 
place  of  our  original 
preferences.  So,  Extraverts 
never  become  Introverts,  and 
vice  versa.  (Kroeger  and 
Thuesen,  1988:11) 


The  eight  dichotomous  pairs,  are 
Extraversion  (E)-  Introversion,  (1)  Sensing  (S) 
-  Intuitive  (N),  Thinking  (T)  -  Feeling  (F), 
and  Judging  (J)  -  Perceiving  (P). 
Combinations  of  the  different  functions 
result  in  one  of  16  distinct  and  separate 
personality  types,  i.e.  ISTJ,  ENFP,  etc. 

The  Myers-Briggs  Type  Indicator 

The  Myers-Briggs  Type  Indicator 
(MBTI)  was  developed,  over  a  period  of 
twenty  years,  by  Isabel  Briggs  Myers  and 
Katherine  Briggs  to  specifically  carry  Carl 
Jung's  theory  of  psychological  type  into 
prachcal  applications.  The  development  of 
the  MBTI  survey  culminates  a  lifetime  of 
observation  and  research  by  Isabel  Briggs 
Myers  in  the  area  of  personality  types 
(Lawrence,  1982:5).  The  main  purpose  of 
the  MBTI  is  summarized  by  Isabel  Briggs 
Myers  and  Mary  McCaulley  in  their 
publication.  Manual:  A  Guide  to  the 
Development  and  Use  of  the  Myers-Briggs  Type 
Indicator. 

The  aim  of  the  MBTI  is  to 
identify,  from  self-report  of 
easily  recognized  reactions,  the 
basic  preferences  of  people  in 
regard  to  perception  and 
judgment,  so  that  the  effects  of 
each  preference,  singly  and  in 
combination,  can  be  established 
by  research  and  put  to  practical 
use.  (Myers  and  McCaulley, 

1985:1) 

There  are  two  reasons  why  the  MBTI 
was  chosen  as  a  survey  instrument  in  this 
research.  The  first  reason  involves  the 
theory  of  personality  type  which  proposes 
that  an  individual's  attitudes  and  behaviors 
are  identifiable  by  the  way  they  judge  and 
perceive  the  world.  Since  one's  choice  of  a 


279 


particular  negotiation  tactic  or  strategy  is 
related  to  an  individual's  judgment  and 
perception  process,  it  may  be  reasonable  to 
postulate  a  correlation  between  the  choice 
of  a  particular  negotiation  tactic  or  strategy 
and  one's  personality  type  may  exist.  If 
such  a  correlation  exists,  the  possible 
benefits  could  be  significant.  Considering 
the  negotiation  process  alone, 
understanding  the  relationship  of 
personality  type  and  negotiation  tactics  and 
strategies  may  allow  the  process  to  be  more 
completely  understood  if  we  could  predict, 
or  at  least  understand,  why  the  opposing 
negotiator  is  acting  or  behaving  the  way  he 
or  she  is. 

The  other  reason  the  MBTI  was 
selected  for  use  in  this  research  is  that  it 
provides  a  survey  instrument  that  has  been 
fully  tested,  refined,  and  proven  valid  and 
reliable  over  time.  Myers  and  McCaulley 
provide  a  history  of  the  actual  construction 
of  the  MBTI  in  their  MBTI  Manual  (Myers 
and  McCaulley,  1985:140-146).  Their 
manual  also  explains  that  the  MBTI's 
internal  reliability,  proven  through  the  use 
of  the  split-half  technique  and  test-retest 
correlations,  is  well  established  as  an 
acceptable  measurement  for  use  in  research 
(Myers  and  McCaulley,  1985:164-174). 
The  validity  of  the  MBTI  has  consistently 
been  proven  over  many  years  by  showing 
that:  the  MBTI  scores  correspond  favorably 
to  other  survey  instruments  that  measure 
Jungian  constructs,  behavior  of  the  MBTI 
types  is  in  concert  with  predicted  MBTI 
type  theory,  and  knowledge  of  type 
differences  contributes  to  the 
understanding  of  other  issues  of 
psychological  importance  (Myers  and 
McCaulley,  1985:175-223). 

The  Relationship  of  Personality  Type  and 
Preferred  Negotiation  Tactics  and 
Strategies 


Johnstone  surveyed  249  out  of  a 
possible  508  contracting  officers  and  price 
analysts  in  the  1102  job  series  employed  at 
the  Aeronautical  Systems  Division  (ASD)  at 
Wright-Patterson  Air  Force  Base,  Ohio. 
The  survey  contained  two  parts,  the  MBTI 
and  a  nine  page  questionnaire  concerning 
negotiation  tactics  and  strategies.  The 
negotiation  questionnaire  listed  ten 
negotiation  strategies  that  the  respondents 
ranked  by  order  of  preference.  Likewise, 
33  negotiahon  tactics  were  presented 
whereby  the  respondent  ranked  their  top 
five  preferences  along  with  listing  the  top 
five  tactics  used  by  their  opponents 
(Johnstone,  1986:57-60). 

Results  of  the  survey  were  composed 
of  responses  from  ninety-nine  participants 
of  which  69  were  male  and  30  were  female. 
The  MBTI  results  were  paired  with  the 
corresponding  negotiation  questionnaire 
responses  and  statistically  analyzed  for 
correlation.  While  the  results  of  the 
statistical  analysis  failed  to  establish  a 
correlation  between  preferred  negotiation 
tactics  and  strategies  and  personality  type, 
it  did  show  that  the  sample's  type 
distribution  was  statistically  different  from 
the  type  distribution  in  the  general 
population.  Johnstone  noted  a 

preponderance  of  ISTJ  (38.4%),  ESTJ 
(20.2%),  and  ENTJ  (8.1%)  in  her  sample 
population  (Johnstone,  1986:xi-xii). 

Johnstone  attributes  the  lack  of 
correlation  between  the  negotiation  tactics 
and  strategies  and  personality  types  to  the 
negotiation  questionnaire  which. 

When  reviewed  in  the  light  of 
the  behavioral  sciences' 
findings  in  the  conflict 
literature,  all  belonged  to  the 
win-lose  or  competitive 
approach.  By  failing  to  offer  a 
range  of  choices  that  would 
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have  allowed  expression  of  the 
differing  preferences  of  the 
psychological  types,  the 

questionnaire  limited  the 
results  to  competitive  solutions 
only.  (Johnstone,  1986:118) 

Johnstone  also  offered  the  possibility  that 
the  heavily  regulated  internal  and  external 
environments  of  the  federal  acquisition 
workforce  may  have  influenced  the  way 
the  respondents  answered  the  survey 
(Johnstone,  1986:119), 

Methodology 

A  mail  survey  was  chosen  to 
accomplish  the  objective  of  establishing  an 
information  base  concerning  personality 
types  and  preferred  negotiahon  strategies 
and  tactics.  The  survey  instruments  for  this 
research  consisted  of  the  Myers-Briggs 
Type  Indicator  (MBTI)  and  a  questionnaire 
on  preferred  negotiation  strategies  and 
tactics  originally  designed  by  Catlin  and 
Faenza  (Catlin  &  Faenza:23-24)  and 

subsequently  modified  by  the  current 
research  team,  in  which  the  survey 
respondents  were  asked  to  indicate,  using  a 
five  point  Likert  scale,  how  often  they  used 
a  particular  negotiation  tactic  or  strategy  (1 
=  Never,  5  =  Always).  Various  statistical 
measures  were  used  to  answer  the  research 
questions.  Demographic  results  were 
arrived  at  using  simple  mean,  standard 
deviahon,  and  median  scores.  A 
comparison  to  the  SRI  International  Values 
and  Lifestyle  Program  Survey  (VALS) 
database  was  conducted  using  a  standard 
chi-squared  statistic  to  determine  statistical 
differences.  A  comparison  of  personality 
types  and  major  sub-types  for  government 
to  industry  was  reported.  Some  of  the 
rci.carch  questions  were  answered  by  a 


comparison  of  means,  in  a  test  of 
hypothesis  (Z  -  test). 

Results 

A  total  of  2000  surveys  were  mailed 
to  National  Contract  Management 
Association  members  in  April  1992.  Of  the 
original  2000  surveys  mailed,  627  usable 
were  returned  for  a  return  rate  of  31.35%. 
In  all  instances  where  the  data  are  stratified 
by  employer,  the  39  respondents  that 
answered  'Other'  to  the  question  regarding 
employer  were  excluded  from  analysis,  and 
the  sample  size  reduced  to  588  instead  of 
627. 

The  demographics  portion  of  the 
survey  consisted  of  eight  questions:  Age, 
gender,  ethnic  origin,  employer,  total 
number  of  years  in  contracting,  highest 
level  of  formal  education  attained,  total 
number  of  hours  of  formal  training  in 
negotiations,  and  percentage  of  time  spent 
conducting  and  managing  contract 
negotiations. 

Analysis  of  the  age  of  the 
respondents  shows  that  19.9%  (125)  of  the 
survey  respondents  are  35  years  of  age  or 
younger,  leaving  80.1%  (502)  of  the 

population  36  years  old  or  older . 

Males  represented  66.7%  (418)  of  the 
survey  respondents,  compared  with  33.3% 
(209)  female  .  The  ethnic  makeup  of  the 
survey  respondents  was  reported  as  93.3% 
(585)  Caucasian,  2.4%  (15)  Black,  2.2%  (14) 
Hispanic,  0.5%  (3)  Oriental,  and  1.6%  (10) 
reporting  Other  .  The  survey  showed  that 
56.9%  (357)  of  the  survey  respondents  were 
employed  in  private  industry,  34.3%  (215) 
employed  by  the  federal  government,  1.6% 
(10)  employed  by  state  government,  1.0% 
(6)  employed  by  local  government,  6.2% 
(39)  reported  being  employed  by  other  than 
the  aforementioned  employers.  The 
majority  (59,0%)  of  contract  negotiators 
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indicated  that  they  possessed  more  than  10 
years  of  contracting  experience,  with  22.5% 
(141)  having  11  to  15  years  of  experience, 
23.6%  (148)  having  16  to  25  years  of 
experience,  and  12.9%  (81)  having  more 
than  25  years  of  experience.  This  left  41% 
with  less  than  10  years  experience;  26.5% 
(166)  having  between  5  and  10  years  of 
experience,  and  14.5%  (91)  having  less  than 
5  years  of  experience  . 

The  education  level  of  the  survey 
respondents  is  as  follows:  3.7%  (23) 

possessed  a  high  school  diploma,  12.1% 
(76)  had  some  college,  less  than  a 
baccalaureate  degree,  38.1%  (239)  had  a 
bachelor's  degree,  43.4%  (272)  had  a 
graduate  or  professional  degree  (i.e.  J.D.) 
degree,  and  2.2%  (14)  possessed  a  doctoral 
degree.  0.5%  (3)  did  not  respond  to  the 
question. 

The  majority  (53.1%)  of  the 
negotiators  who  responded  indicated  that 
they  had  received  more  than  40  hours  of 
formal  training  in  contract  negotiation. 
28.9%  (181)  of  the  respondents  indicated 
that  they  had  received  more  than  80  hours 
of  formal  training.  0.3%  (2)  respondents 
did  not  indicate  their  level  of  training. 

Thirty  six  percent  (226)  of  the 
respondents  indicated  that  they  spent  more 
than  25%  of  their  time  in  their  current 
position  either  negotiating  or  managing 
contract  negotiations.  45.9%  (288)  indicated 
that  they  spend  less  than  25%  of  their  time 
in  their  current  position  negotiating  or 
managing  contract  negotiations.  18.0% 
(113)  indicated  that  they  did  not  currently 
negotiate  in  their  present  position.  0.3%  (2) 
did  not  respond  to  the  question.  A 
composite  survey  respondent  would  be  a 
male  Caucasian,  between  36-45  years  old, 
employed  by  private  industry  with  11  to  15 
years  of  experience  in  the  contracting  field, 
possess  a  master's  degree,  have  received 
between  20  and  40  hours  of  formal  training 


on  contract  negotiations,  and  spends  less 
than  25%  of  his  time  in  his  current  position 
negotiating  or  managing  the  negotiation  of 
contracts. 

Because  the  primary  focus  of  this 
research  is  the  relationship  between 
government  and  industry,  the  data  were 
stratified  by  employer  and  analyzed  for 
differences.  Only  two  professional 
demographics  questions  showed  a 
statistically  significant  difference  at  the 
p<.01  level.  Industry  contract  negotiators 
indicated  that  they  have  more  years  in  the 
contracting  profession  (mean  scores 
Industry  =  3.02,  standard  deviation  =  1.28; 
Government  =  2.74,  standard  deviation  = 
1.19),  while  government  negotiators 
indicated  that  they  had  received  more 
formal  training  than  their  industry 
counterparts  (mean  scores  Industry  =  2.73, 
standard  deviation  =  1.60;  Government  = 
3.15,  standard  deviation  =  1.55). 

Research  Question  One.  Witat 
negotiation  tactics  and  strategies  are  most 
frequently  used  by  contract  negotiators? 

The  data  indicate  that  all  of  the 
negotiation  tactics  and  strategies  are  used 
to  some  extent  The  ten  most  often  used 
tactics  are  (in  descending  order  of 
frequency  of  use,  i.e.  Allow  face  saving  exit 
was  most  frequently  used,  the  next  was  less 
frequently  used,  etc.):  Allow  face  saving  exit; 
Split  the  difference;  Pick  and  choose  the  best 
deals;  Bogey  -  constrained  by  budget  limits; 
Refer  to  your  side's  generosity;  Call  frequent 
caucuses;  Belabor  fair  and  reasonable;  Massage 
opponent's  ego;  Refer  to  other  side's  past  poor 
performance;  and  Good  Guy  -  Bad  Guy  Roles. 
It  should  be  noted  that  the  data  show 
negotiators  as  a  whole  do  not  often  use 
tactics  that  could  be  construed  as  negative 
or  unethical  in  nature.  The  data  also  show 
that  there  is  little  or  no  difference  between 
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government  and  industry  negotiators  in 
terms  of  what  tactics  they  tend  to  shy  away 
from.  The  low  standard  deviations 
reflected  of  the  least  frequently  used  tactics 
also  show  relative  agreement  among  the 
survey  respondents  as  to  the  unappealing 
nature  of  the  tachc.  The  ten  least  frequently 
used  tactics  are  (in  ascending  order  of 
frequency  of  use,  i.e.  Adjust  the  thermostat 
was  used  least  often,  the  next  tactic  used 
more  that  the  previous,  etc.);  Adjust  the 
thermostat;  Deliberately  leave  errors  in  offers; 
Personal  attack;  Deliberately  expose  notes  or 
working  papers;  Reverse  auctioning;  'Off  the 
record'  discussions;  Negotiate  with  limited 
authority;  and  ask  for  excessive  amounts  of  data. 

The  survey  respondents  indicated 
that  they  used  the  strategies  in  the 
following  order  (in  decreasing  frequency 
order  of  frequency  of  use)  Win-Win, 
Statistics  -  figures  don't  lie,  Participation, 
Coverage  -  Bottom  Line,  Step-by-Step, 
Combination  -  the  big  pot.  Definite  Action  - 
testing  the  waters.  Limits,  Patience  -  buying 
time  or  stalling.  Surprise,  and  Reversal. 

Although  the  Win-Win  strategy  was 
shown  to  be  the  strategy  most  often  used, 
the  data  clearly  show  that  all  of  the 
strategies  are  used  to  some  extent. 

Research  Question  Two.  What  is  the 
personality  type  composition  of  contract 
negotiators?  The  distribution  of 
personality  types  as  measured  by  the 
Myers-Briggs  Type  Indicator  (MBTI)  are 
shown  at  Table  1. 

The  survey  respondents  were 
compared  to  the  SRI  International  Values 
and  Lifestyle  Program  Survey  to  determine 
if  the  survey  sample  was  similar  to  that  of 
the  general  population. 

When  the  data  were  stratified  by 
employer,  the  test  shows  the  two 
populations  are  statistically  different 


Therefore,  one  can  conclude  that  the 
composition  of  personality  type  as 
measured  by  the  MBTI  for  the  contract 
negotiators  that  responded  to  this  survey  is 
not  the  same  as  the  that  of  the  general 
population.  Further  analysis  provided  the 
personality  type  composition  of 
government  versus  private  industry 
contract  negotiators.  It  is  interesting  to  note 
that  the  government  strata  had  a 
proportionately  higher  amount  of  females 
in  the  population  than  the  industry  strata, 
although  no  explanation  is  offered.  A  chi- 
squared  analysis  was  performed  using  the 
larger  industry  strata  as  the  base  to  which 
the  government  strata  was  compared.  A 
chi-squared  value  of  9.1086  was  received 
for  males,  and  23.3375  for  females, 
indicating  no  significant  differences 
between  the  two  strata,  at  the  p  <  .001  level 
of  significance. 

Research  Question  Three.  Is  there  a 
differettce  in  tiegotiation  strategies  or 
tactics  used  by  government  and  industry 
negotiators?  There  was  not  a  significant 
difference  between  the  order  of  use  of  the 
particular  tactics  and  strategies.  The  data 
did  show  that  the  industry  negotiators 
tended  towards  a  higher  level  of  use  of 
specific  tactics  than  their  government 
counterparts. 

While  there  is  not  a  significant 
difference  in  the  strategies  used  by  both 
government  and  industry  negotiators, 
again  the  data  suggests  that  while 
government  negotiators  use  negotiation 
strategies,  they  do  so  at  a  level  lower  than 
that  of  the  industry  negotiators.  It  should 
be  noted  that  the  level  of  use  for  all  of  the 
strategies  was  higher  than  that  of  tactics, 
regardless  of  employer.  This  despite  the 
definihon  provided  to  the  survey 
respondents  that  defined  negotiation 
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Table  1 


DISTRIBUTION  OF  PERSONALITY  TYPES  OF  CONTRACT 
NEGOTIATORS  AS  MEASURED  BY  THE 
MYERS-BRIGGS  TYPE  INDICATOR  (MBTI) 


(n  =  627) 


ISTI 

ISFI 

INFI 

INTI 

N  =  151 
%  =  24.1 

N  =  3 
%  =  0.5 

N  =  3 
%  =  0.5 

N  =  33 
%  =  5.3 

ISTP 

ISFP 

INFP 

INTP 

N=101 
%  =  16.1 

N  =  5 
%  =0.8 

N  =  7 
%  =  1.1 

N  =  63 
%  =  10.0 

ESTP 

ESFP 

ENFP 

ENTP 

N  =  79 
%  =  12.6 

N  =  20 
%  =  3.1 

N  =  10 
%  =  1.6 

N=47 
%  =  73 

ESTJ 

ESFJ 

ENFI 

ENTJ 

N  =83 
%  =  13.2 

N  =  7 
%  =  1.0 

N  =  1 
%  =  0.16 

N  =  14 
%  =  2.2 

NOTE:  Percentages  rounded  off  to  the  nearest  tenth. 


strategy  as  "an  organized  plan  or  approach  to 
negotiations  from  an  overall  perspective  that 
may  be  composed  of  one  or  more  than  one  tactic” 
(Catlin  and  Faenza,  1985:99).  A  test  of 
hypothesis  was  conducted  to  determine  if 
statistically  significant  differences  exist 
between  the  two  groups  in  terms  of  how 
often  each  group  used  the  respective  tactics 
and/or  strategies.  The  same  test  statistic 
(paired  z-  test)  was  used  as  in  previous 
analyses.  The  data  was  stratified  by 
employer,  excluding  respondents  who 
answered  Other  to  the  question. 
Respondents  who  reported  Federal,  Local, 
or  State  Government  were  grouped  as  the 


government  negotiators,  and  respondents 
who  answered  Industry  were  grouped  as 
such.  The  analysis  shows  that  25 
significant  differences  between  the  groups 
exist,  20  tactics  questions,  and  5  strategy 
questions.  One  would  expect  to  find  that 
government  negotiators  would  use  Appeal 
to  Patriotism,  Impose  No  Smoking  Rule,  and 
Low  Ball  tactics  more  often  than  industry 
negotiators  due  to  the  nature  of  the 
respective  environments  (federal  versus 
commercial  contracting),  and  that  industry 
negotiators  would  use  Pick  and  Oioose  the 
Best  Deals  and  High  Ball  tactics  more  often. 
This  is  in  fact  borne  out  by  the  data. 
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Discussion  of  the  Conclusions 

The  objectives  of  the  research  were 
to  determine: 

A)  what  negotiation  tactics  and 
strategies  are  used,  and  how  often; 

B)  the  relationship  between 

government  contract  negotiators 
and  industry  negotiators  in  terms  of 
what  tactics  and  strategies  are  used 
by  each  group,  and  how  often; 

C)  if  government  contract 

negotiators,  as  a  population,  differ 
from  their  industry  counterparts  in 
terms  of  psychological  composition. 

The  research  objectives  were  met  via  the 
answering  of  the  research  questions.  It  was 
established  that  various  tactics  and 

strategies  are  used  by  contract  negotiators, 
and  that  they  are  used  in  varying 
frequencies.  It  was  determined  that 
industry  contract  negotiators  report  more 
frequent  use  of  negotiation  tactics  and 
strategies  than  their  government 
counterparts  (that  is  to  say  industry 
contract  negotiators  recorded  more 
responses  in  the  Sometimes,  Frequenthj,  and 
Always  range  than  goverriment 
negotiators).  It  was  also  established  that 
the  government  and  industry  negotiators 
are  similar  in  terms  of  personality  type 
distribution  (psychological  composition). 

It  can  be  seen  that  contract 
negotiators  use  the  identified  negotiation 
tactics  and  strategies.  Conversely,  the 
sample  indicated  that  they  did  not  use 
other  negotiation  tactics  as  often.  Of  the 
negotiation  strategies,  the  sample  indicated 
that  all  were  used  more  than  Seldom,  and  in 
fact  all  but  two  were  used  more  than 
Sometimes.  The  negotiation  strategies  most 


often  used  are  Win-Win  ai.  i  Statistics,  with 
the  least  used  being  Reversal  and  Surprise. 

looking  at  the  personality  type 
distribution  of  contract  negotiators,  it  was 
found  that  the  negotiators  have  a  unique 
distribution,  heavily  favoring  the  ST 
(Sensing-Tliinking)  functional  grouping, 
with  a  marked  under-representahon  of 
Feeling  (F)  types.  This  distribution  is 
similar  to  that  found  in  Johnstone's  study 
(Johnstone,  1986).  The  sample  was 
compared  to  the  general  population  as 
personified  by  the  SRI  database,  and  found 
to  be  statistically  different  from  the  general 
population. 

When  analyzing  the  differences 
between  goveriunent  and  industry 
negotiators  in  terms  of  frequency  of  use  of 
negotiation  tactics  and  strategies,  it  was 
established  that  industry  negotiators  used 
tactics  and  strategies  more  often  than  their 
government  counterparts  across  the  board. 
Ranking  the  mean  scores  of  the  ten  most 
frequently  used  tactics  and  strategies  by 
both  strata  yielded  little  significant 
difference  in  terms  of  which  tactics  were 
used  most  often  by  the  two  strata. 
Statistically  significant  differences  were 
established  between  government  and 
industry  negotiators  on  20  out  of  33  tactics, 
and  5  out  of  eleven  strategies,  leading  to 
the  conclusion  that  the  two  groups  are  not 
the  same  in  terms  of  now  often  each  tactic 
or  strategy  is  used. 

Discussion 

Overall,  the  research  objectives  were 
satisfied.  The  purpose  of  this  research  was 
to  determine  differences  between 
government  and  industry  contract 
negotiators  in  terms  of  what  negotiahon 
tactics  and  strategies  are  used,  how  often 
they  are  used,  and  the  relationship  of 
personality  type,  as  measured  by  the 
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Myers-Briggs  Type  Indicator,  to  use  of 
negotiation  tactics  and  strategies.  This 
research  established  the  fact  that 
negotiation  tactics  and  strategies  are  used 
by  contract  negotiators,  in  ooth 
government  and  industry.  The  impact  of 
the  use  of  the  various  tactics  and  strategies, 
however,  is  unknown.  A  study  to  examine 
the  relationship  between  tactic  or  strategy 
used  and  negotiation  outcome  would 
provide  invaluable  information  to  contract 
negotiators.  It  would  then  be  possible  to 
pick  appropriate  negotiation  tactics  or 
strategies  that  would  ensure  a  more 
favorable  outcome. 

The  National  Contract  Management 
Association  has  made  large  inroads  into  the 
education  of  its  members  in  terms  of 
negotiation  and  the  use  of  tactics  and 
strategies.  However,  as  this  research 
showed,  some  tactics  are  used  that  are 
clearly  not  in  the  best  interests  of  either 
party  in  a  negotiation,  such  as  Personal 
Attack  and  Embarrass  Your  Opponent.  While 
the  mean  scores  indicating  frequency  of  use 
of  the  particular  tactics  were  low  (less  than 
2.0),  there  were  individual  contract 
negotiators  that  indicated  that  they  used 
the  tactic  more  than  Sometimes.  This 
indicates  that  some  NCMA  members  are 
choosing  to  ignore  the  valuable  training 
provided  by  NCMA  in  their  educational 
workshops,  or  some  members  did  not 
partake  of  the  training.  In  any  case,  it  is  in 
the  best  interests  of  the  NCMA  to  continue 
its  aggressive  stance  on  education  in  terms 
of  negotiation.  A  well-educated 
membership  reflects  on  the  organization. 

Congressional  and  Presidential 
Commissions  findings  to  the  contrary,  this 
research  found  that  government  contract 
negoHators,  while  less  experienced  than 
their  industry  counterparts,  are  better 
trained  in  the  'science'  of  negotiation.  This 


research  also  found  that  government 
negohators  are  likely  to  use  certain 
negotiation  tactics  and  strategies,  in 
particular  those  that  are  government 
unique,  such  as  Appeal  to  Patriotism,  High 
Ball  versus  Low  Ball  tactics,  along  with  the 
Win-Win  and  Statistics  strategies.  However, 
with  the  exception  of  a  few  of  the 
negotiation  tactics  and  strategies  that  are 
specific  to  the  government  such  as  those 
listed  above,  government  negotiators  across 
the  board  used  tactics  and  strategies  less 
frequently  than  their  industry  counterparts. 
It  would  benefit  government  contracting 
agencies  if  it  were  possible  to  identify  what 
tactics  and  strategies  are  effectively  used  in 
certain  scenarios,  and  also  to  identify  their 
impact  on  final  contract  settlement  We 
also  strongly  recommend  that 
organizations  use  the  Myers-Briggs  Type 
Indicator  in  building  their  negotiation 
teams.  This  research  established  that 
negotiation  tactics  and  strategies  differ 
between  personality  functional  types. 
Negotiators  will  be  better  off  in  conducting 
a  negotiation  if  the  negotiator  understands 
his  or  her  teammates  in  terms  of 
preferences,  and  can  work  towards  more 
satisfying  conclusions  in  the  negotiation 
process. 

Captain  Hebert  and  Mr.  Meade  conducted  this 
research  under  the  advisement  ofDrs.  Campbell, 
Pursch  and  Graham  while  graduate  students  at 
the  Air  Force  Institute  of  Technology  in  1991- 
92.  Their  award-winning  thesis,  "Examination 
of  Psychological  Type  and  Preferred  Negotiation 
Tactics  and  Strategies  of  Contract  Negotiators" 
is  on  file  with  the  Defense  Technical 
Information  Center. 
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EXPERT  PRICING  SYSTEMS 
by 

Richard  J.  Hernandez,  CPCM 
Vendor  Programs,  Diversity  Development  Department 
U.S.  Postal  Service 


Abstract.  This  paper  discusses  the 
use  of  rule-based  expert  systems,  a 
form  of  artificial  intelligence  (Al),  as  a 
means  to  improve  the  speed,  quality, 
and  accuracy  of  the  contract  pricing 
function.  Expert  systems  are  capable 
of  improving  efficiency,  as  well  as 
effectiveness,  since  they  possess  an 
additional  knowledge  element  not 
found  in  electronic  spreadsheets.  The 
additional  knowledge  element  is 
essential  given  "the  complexity  of 
today's  business  environment  is 
increasing  faster  than  human  skill." 
(1)  Rule-based  expert  systems  work 
by  asking  human  users  a  series  of 
questions,  which  in  turn  use  rules, 
historical  data,  and  computer  logic  to 
help  humans  solve  complex  problems. 
There  are  many  potential  pricing- 
related  microcomputer-based 

applications  for  this  new  type  of 
technology  such  as:  contract 

negotiations  (helping  buyers  and 
sellers  set  targets),  budgeting, 
feasibility  analyses,  and  establishing  a 
reliable  corporate  pricing  data  base. 

Introduction.  Pricing  is  an  integral  part 
of  the  contract  management  process, 
the  results  being  used  to  determine  if 
a  "fair  and  reasonable"  price  or  "best 
value"  has  been  achieved.  However, 
the  quality  of  a  cost/price  analysis 
can  vary  considerably,  depending  on 
the  skill,  experience,  and  knowledge 
of  the  individual  estimator  or 
cost/price  analyst.  Pricing  is  done 
using  one  or  more  of  the  following 


techniques:  parametrics  (rough 

yardsticks  such  as  correlating  cost  to 
weight  of  an  aircraft),  analogies 
(comparing  prices  of  similar 
items/systems),  using  past  bids, 
and/or  engineering  estimates. 
Developing  a  sound  corporate  pricing 
expertise,  for  a  private  company  or  a 
government  Agency,  can  be  difficult 
given  the  extensive  training  and  time 
required  for  an  individual  or  work 
group  to  achieve  a  satisfactory  level 
of  proficiency. 

The  most  common  pricing  automation 
tool  in  use  today  is  the  electronic 
spreadsheet.  These  systems  improve 
efficiency  by  automating  the 
mechanical  calculation  of  rows  and 
columns  of  numbers.  Some  advanced 
systems  also  provide  multi¬ 
dimensional  computational  capability, 
regression  analysis,  forecasting,  audit 
trail  capability,  as  well  as  impressive 
graphical  displays.  Many 

spreadsheets  are  able  to  exchange 
data  with  each  other  as  well  as  with 
other  applications  such  as  data  base 
utilities.  However,  unlike  expert 
pricing  systems,  spreadsheets  do  not 
improve  the  effectiveness  of  the 
cost/price  estimation  process. 

Overview  of  Expert  System 
Technology.  Expert  systems  were 
first  conceived  in  the  late  1 950s.  The 
term  "expert  system"  was  derived 
since  these  traditional,  rule-based 
systems  perform  in  a  similar  manner 
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as  human  subject-area  experts  in  that 
they  give  advice,  make  predictions, 
and  solve  problems.  Recently,  case- 
based  reasoning  tools  have  begun 
making  advances  in  areas  where  rule- 
based  expert  systems  are  not 
applicable.  Since  there  has  been  little 
application  of  case-based  reasoning  to 
date,  this  paper  will  deal  exclusively 
with  rule-based  expert  systems. 

Expert  systems  are  composed  of  two 
essential  elements.  First,  there  is  a 
knowledge  base,  which  is  usually 
limited  to  a  specific  area  or  domain. 
Second,  there  is  an  inference 
"engine"  or  system  which  performs 
the  actual  problem  solving.  Expert 
systems  use  heuristics,  or  rules  of 
thumb,  when  they  operate.  It  is 
important  to  note  "Expert  systems  are 
based  on  the  idea  that  rules  are  an 
effective  way  to  tell  computers  how 
to  do  certain  kinds  of  things  that 
people  do."  (2)  Unlike  conventional 
computer  programming  languages 
such  as  FORTRAN,  COBOL,  or  BASIC, 
expert  systems  use  advanced 
programming  languages  such  as  LISP, 
PROLOG,  and  C  which  permit  the 
creation  of  new  algorithms  to  solve  a 
variety  of  problems.  Conventional 
programming  languages  do  not  have 
this  inherent  flexibility.  A  further 
comparison  of  expert  systems  versus 
traditional  computer-based 
information  systems  is  provided 
below: 

Expert  systems  are  in 
general  less  well  defined 
than  information 
systems,  and  their  role  is 
less  precise.  They  are 
'open'  and  need  to  be 
able  to  respond  to 


changing  environments, 
they  handle  incomplete 
information  and 
imprecise  information, 
they  cannot  be  fully 
tested  and  it  is  difficult 
to  define  exactly  how 
they  behave  even  after 
they  have  been  built.  In 
many  instances  the  users 
of  expert  systems  are 
professional  people  who 
are  used  to  making 
important  decisions  on 
the  basis  of  incomplete 
or  imprecise  information. 

The  systems  are  often 
intended  for  judgmental 
tasks  which  are  central 
to  professional  jobs.  (3) 

Expert  systems  take  advantage  of  the 
computer's  ability  to  use  symbols  as 
well  as  make  numerical  calculations. 
Knowledge  engineers  help  bridge  the 
gap  between  programmers  and  users 
(domain  specialists)  when  developing 
expert  systems. 

The  advent  of  a  new  and  low-cost 
generation  of  microcomputers  in  the 
late  1980s  and  early  1990s  has  given 
rise  to  a  proliferation  of  expert  system 
applications.  Consider  the  following: 

Tasks  such  as  figuring 
out  how  to  turn  on  a 
lamp  require  so  many 
rules  and  so  many  facts 
that  computers  big 
enough  for  worthwhile 
reasoning  were  not 
available  until  the  early 
1 980s.  Throughout  the 
1970s  it  cost  several 
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million  dollars  to  buy  a 
computer  big  enough  to 
run  even  the  smallest  Al 
programs,  so  few 
organizations  had  any 
interest  in  Al  at  all.  (4) 

In  fact  many  vendors  sell  expert 
system  kits  which  are  know  as 
"shells."  These  commercially- 
available  shells  provide  a  do-it- 
yourself  development  tool  for  expert 
system  application  designers  where 
users  add  the  appropriate  decision 
rules.  It  should  be  noted:  "A  shell 
has  a  rule  system,  a  rule  editor,  and 
tools  for  maintaining  and  debugging 
rules  and  facts."  (5)  Many  of  these 
shells  are  available  for  under  five 
hundred  dollars. 

Benefits  of  using  expert  systems  for 
contract  pricing  are:  increased 
accuracy  of  estimates;  improved 
productivity  of  buyers  and  cost/price 
analysts;  consistency;  reduced 
overhead  costs  since  fewer  people 
are  required  to  prepare  the  cost  or 
price  estimate;  and  increased 
competitiveness.  For  small  firms  the 
above  advantages  could  also  mean 
expert  system  can  help  with  better 
leveraging  personnel  and  knowledge 
resources,  the  result  being  a  leveling 
the  playing  field  between  large  and 
small  businesses.  Expert  systems  can 
also  serve  as  a  teaching  aid  for  new 
contract  specialists  or  cost/price 
analysts.  This  allows  senior  price 
analysts  to  spend  their  time  working 
more  complex  problems  or,  in  the 
case  of  contractors,  using  them  for 
high-skill  jobs,  which  typically  yield  a 
higher  profit  margin  or  involve  a 
greater  risk. 


Applications.  This  section  discusses 
practical  applications  of  real-world 
expert  pricing  systems  in  the  areas  of 
hazardous  waste  cleanup,  logistics, 
and  weapon  system  acquisition. 
Examples  of  some  of  the  systems  this 
research  effort  will  examine  are:  1) 
Logistics  Expert  System  Advisor,  2) 
Superfund  Cost  Estimating  Expert 
System,  3)  Cost  of  Remedial  Action 
Model,  and  4)  Logistics  Planning  and 
Requirements  Simplification  System. 

Logistics  Expert  System  Advisor 
(LESA).  This  microcomputer-based 
system,  developed  by  Intellogistics  in 
Columbus,  OH,  helps  managers 
determine  the  feasibility  of  using  an 
expert  system  for  a  particular  logistics 
application.  LESA  provides  a  rough 
order  of  magnitude  cost  estimate  as 
well  as  return  on  investment.  Users 
must  answer  either  fill-in-the  blank  or 
multiple  choice  questions. 

There  are  three  specific  types  of  cost 
estimates  for  the  logistic  expert 
system:  development,  fielding  and 

maintenance.  All  costs  are  rounded 
to  the  nearest  thousand  dollar  level. 
Development  costs  are  broken  down 
by  personnel,  hardware,  and 
software.  Personnel  costs  are  the 
product  of  the  number  of  persons 
required  to  develop  the  expert 
system,  their  salary,  labor  burden 
(such  as  benefits  and  taxes),  and  the 
number  of  years  required  to  develop 
the  system.  The  user  manual 
provides  an  additional  explanation  on 
how  development  costs  are 
calculated: 
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Development  costs  are 
fundamentally  the  sum  of 
the  sunk  costs  plus  the 
product  of  the  variable 
costs  over  time.  The 
sunk  costs  are  the  expert 
system  tool  cost  and 
hardware  costs  (if  any), 
while  the  variable  costs 
are  the  personnel  costs. 

The  development  costs 
which  LESA  displays  do 
not  include  the  costs  of 
coding  any  external 
interfaces  or  databases 
nor  the  purchase  fees  of 
any  required  database 
software  which  the 
expert  system  may 
require.  (6) 

The  guide  also  notes:  "The  cost  of 
the  expert  system  software  is  based 
on  the  least  expensive  tool  known  to 
LESA  that  is  compatible  with  the 
hardware  and  software  (operating 
system  and  database)  constraints  you 
set  forth  when  answering  the 
questions  about  hardware,  operating 
system,  and  database  requirements.” 
(7)  Additional  notes  about 

determining  development  costs  are 
that  personnel  costs  are  determined 
by  experience  levels  of  the 
developers.  Users  have  the  option  to 
change  salary  levels  depending  on  the 
situation.  Development  costs  also 
include  the  preparation  of 
documentation.  Fielding  costs  are 
also  determined  based  on  the  least 
expensive  computer  configuration; 
they  also  include  such  items  as  travel 
and  installation.  Maintenance  costs 
are  based  on  the  number  of  users 
supported,  hardware,  and  software 


costs.  LESA  also  identified  the  time 
period  associated  with  the 
maintenance  costs.  The  LESA  guide 
identifies  a  key  limitation:  "One 

limitation  of  LESA  is  that  it  correctly 
considers  only  one  configuration  at  a 
time."  (8) 

Besides  providing  a  cost  estimate, 
LESA  also  provides  a  schedule 
estimate,  a  recommendation  on 
whether  or  not  to  proceed,  the 
probability  that  the  expert  system  will 
success,  and  hardware/software 
recommendations.  Other  useful 
features  are  an  "accelerator"  key 
which  speeds  up  LESA  functions  as 
well  as  the  ability  to  saves  a 
consultant  with  the  expert  system 
and  resume  it  at  a  later  time. 

Superfund  Cost  Estimating  Expert 
System  (SCEES).  This 

microcomputer-based  system  is  used 
by  the  U.S.  Environmental  Protection 
Agency  (EPA)  to  estimate  hazardous 
waste  clean  up  costs  associated  with 
the  "Superfund"  effort.  Specifically, 
SCEES  is  used  to  estimate  Remedial 
Investigation  and  Feasibility  Study 
phases  of  toxic  waste  site  clean  up  at 
landfills  and  lagoon/waste  pits.  The 
system  was  developed  by  an  EPA 
contractor  using  the  Nexpert  Object 
expert  system  shell.  PC  Al  magazine 
describes  a  sample  SCEES  operation: 

SCEES  dynamically 

creates  objects  during 
processing.  This  is 
especially  useful  when 
dealing  with  well 
installation,  one  of  the 
most  expensive  site 
investigation  activities. 
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Investigators  use  wells  to 
sample  underlying 

aquifers  (underground 
layers  of  water  bearing 
rock,  sand,  or  gravel) 
and  thus  determine  the 
extent  of  groundwater 
contamination. 

Installation  cost  depends 
on  how  deep  and 
through  what  geologic 
materials  (layers  of  sand, 
clay,  rock,  etc.)  well 
must  be  drilled,  and  on 
the  well  casing  material 
(pvc,  stainless  steel).  To 
calculate  this  cost, 

SCEES  constructs  its 
own  model  of  the  site 
geology.  As  it  collects 
the  information  needed 
to  estimate  the  drilling 
subtask,  SCEES 

dynamically  creates 
aquifer  objects 

(representing  layers  of 
geologic  materials)  and 
we// objects.  (9) 

SCEES  makes  its  estimates  using 
heuristic  rules  used  by  human 
experts,  factual  knowledge,  unit 
costs,  cost  trends,  and  past  costs. 
To  further  improve  accuracy,  users 
have  the  capability  to  enter  site- 
specific  data.  SCEES  also  provides 
the  ability  to  examine  cost  cutting 
options  associated  with  toxic  waste 
cleanup  efforts.  Other  data  provided 
by  SCEES  is  a  cleanup  schedule 
estimate  and  risk 

assessment/description.  According  to 
PC  Al,  "SCEES  has  been  delivered  to 
all  ten  EPA  Regions,  as  well  as  to 
groups  in  the  Department  of  Energy 


and  the  U.S.  Navy.  (10)  The 
magazine  also  notes  SCEES  was 
successful  tested  by  independent 
evaluators.  To  Improve  its  future 
utility,  SCEES  may  be  modified  to 
allow  it  to  consider  other  types  of 
cleanup  site  situations  such  as 
buildings  or  tank/drum  sites. 

Cost  of  Remedial  Action  (CORA). 
This  microcomputer-based  system  is 
used  to  estimate  the  cost  of  remedial 
actions  at  EPA  Superfund  sites. 
CORA  has  been  in  use  since  1987, 
with  Version  3.0  being  released  in 
May  1990.  The  system  is  composed 
of  two  basic  parts;  an  expert  system 
and  a  cost  system.  The  expert  part 
uses  the  Level  5  expert  system  shell 
and  the  cost  part  uses  Database  III 
Plus.  The  parts  operate 

independently.  CORA  is  designed  for 
stand-lone  operation  on  a 
microcomputer.  The  CORA  is 
described  in  detail  below: 

The  CORA  cost  system  is 
used  to  develop  order-of- 
magnitude  cost  estimates 
(-30%  to  -1-50%)  for 
hazardous  waste  site 
remediation  following  the 
development  of  response 
action  scenarios,  using 
the  expert  system  or 
other  sources.  The 
CORA  cost  system 
organizes  cost  estimates 
by  site,  operable  unit, 
scenario,  and 

technology.  The  system 
and  the  user  interact  to 
complete  cost  model 
data  requirements  for  a 
site  previously  entered 
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into  the  database  or  for  a 
new  site.  The  CORA 
cost  system  calculates 
capital  and  first-year 
operation  and 

maintenance  (O&M)  cost 
estimates  for  each  of  the 
40  technologies  which 
may  be  selected.  The 
user  may  save  cost 
outputs  to  a  database  for 
subsequent  analysis.  In 
addition,  the  CORA 
model  generates  a  total 
summary  report  for  a  site 
or  operable  unit  for  both 
capital  and  O&M  costs. 

The  summary  report 
includes  costs  incurred 
by  construction  and 

operation  of  individual 
unit  processes  and 
operations,  costs  for 
items  such  as  site 

preparation  and 

administration,  startup, 
permitting  and  legal 
services,  permit  and 

insurance  renewal, 

services  during 

construction,  and  bid  and 
scope  contingencies. 

(11) 

According  to  a  HAZMACON  89 
research  paper,  the  CORA  expert 
system  consists  of  670  hazardous 
waste  cleanup-related  decision  rules 
for  40  technologies.  The  system 
allows  the  user  to  combine 
technologies  as  part  of  the  corrective 
action.  The  cost  system  component 
of  the  CORA  model  contains  40 
primary  cost  modules.  These 
modules  organize  the  cost  estimate 


by  site,  operable  unit,  scenario,  and 
technology.  The  system  uses  default 
values  for  its  cost  models;  these  can 
be  adjusted  depending  on  the  specific 
site  situation.  The  model  contains  the 
following  cost  modules:  Removal, 
Containment,  Treatment,  Disposal, 
Auxiliary  (e.g.,  contingencies  and  site 
preparation/administration),  and 

Miscellaneous  (e.g.,  transportation 
and  user-supplied  costs).  Potential 
future  enhancements  to  CORA  are 
budgeting  forecasts  and  anticipating 
costs  of  new  environmental 
regulations.  (12) 

Logistics  Planning  and  Requirements 
Simplification  System  (LOGPARS). 
This  microcomputer-based  system, 
developed  by  the  Material  Readiness 
Support  Activity  (MRSA)  of  the  U.S. 
Army  Material  Command,  is  used  to 
develop  logistic  plans,  schedules, 
statements  of  work,  warranties,  and 
contract  data  requirements  lists  for 
weapon  systems.  LOGPARS  also 
contains  a  cost  adviser  module,  which 
estimates  the  cost  of  Logistics 
Support  Activity  (LSA)  services.  The 
user  manual  notes: 

The  cost  estimation  is 
done  in  accordance  with 
MRSA  700-11,  Cost 
Estimating  Methodology 
for  Logistics  Support 
Analysis  (CELS A).  The 
Cost  Advisor  implements 
the  CELSA  Guides  cost 
estimating  methodology 
to  estimate  the  cost  of 
LSA  tasks  which  are 
recommended  by 

LOGPARS,  specified  as 
being  contractor 
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executed,  and  are  not 
specified  as  already 
being  completed.  The 
Cost  Advisor  produces  a 
report  which  lists  the 
LSA  task  numbers,  titles, 
man-hour  estimates,  and 
total  cost.  (13) 

To  use  the  Cost  Advisor,  users  must 
define  cost  variables,  change 
"scalers"  (used  to  adjust  man-hour 
estimates),  when  required,  and  then 
request  the  system  perform  its 
calculations.  When  performing 
calculations,  the  system  uses  man¬ 
hour  estimate  tables,  which  the  user 
must  select.  LOGPARS  also  allows 
for  user-defined  scalers.  A  "Screen 
Help"  feature  which  explains  the 
questions  asked  by  the  system. 
Currently,  only  the  cost  of  services 
can  be  estimated.  However,  in  the 
future,  the  LOGPARS  cost  module  will 
include  Integrated  Logistics  Support 
costs. 

Rule-based  expert  systems  are  not  the 
only  form  of  Al  that  may  advance 
contract  and  pricing  activities.  Case- 
based  reasoning  technologies  may 
also  offer  benefit  in  the  near  future. 
A  form  of  case-based  reasoning. 
Associative  Reasoning,  has  been 
applied  to  development  of  a  prototype 
application  enabling  the  selection  of 
estimating  analogs  from  a  historical 
database  of  appropriate  parameters. 

Dr.  F.  N.  Burt,  the  creator  of  the 
patented  technology  applying 
Associative  Reasoning,  states: 

Human  judgment  is  the 
process  of  Associative 


Reasoning  (reasoning  by 
comparison  with  a 
selected  set  of  past 
examples  which  illustrate 
the  logic  appropriate  to 
the  issues  raised  by  the 
current  example)  around 
the  combination  of 
characteristics  that 

makes  a  given  example 
an  exception  to  rules  or 
to  probability.  (14) 

Within  a  presentation  at  the  ISPA 
1990  Annual  International 
Conference,  Dan  Walkovitz  and 
Charlie  Hopkins  describe  this  concept 
in  a  practical  fashion: 

The  Judgment  Processor 
(a  tool  embodying  the 
concept  of  Associative 
Reasoning)  falls  within 
the  category  of  expert 
systems  software. 

Unlike  other  expert 
systems  tools,  however, 
this  software  is  not  rule- 
based.  Rule-based 

software  demands  that 
the  rules  be  defined  by 
the  expert  up  front. 
Experts  do  not,  in  fact, 
even  know  the  rules  that 
they  apply  ... 

Experts  do  not  make 
their  decisions  according 
to  rules.  As  a  matter  of 
fact,  expert  break  rules 
set  by  others.  They  use 
insight  and  intuition,  and 
apply  these  through  the 
use  of  judgment  which 
enables  them  to  make 
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decisions  from  "gut 
feelings"  which  turn  out 
to  be  correct  more  often 
than  similar  feelings  by 
non-experts.  (15) 

This  concept  of  judgment  is,  in  fact, 
captured  by  Judgment  Processor  and 
its  next  generation  product.  Decision 
Maker.  The  ability  of  this  technology 
to  capture  judgment  without  the 
definition  of  rules  and  to  "reason"  or 
extrapolate  beyond  the  cases  it  has 
seen  appears  to  offer  great  potential 
beyond  rule-based  technologies  in  the 
contract  and  pricing  domain. 

Workplace  Implications.  There  are 
several  key  "people"  issues 
associated  the  introduction  of  expert 
pricing  system  technology  into  the 
contract  management  work  place. 
Currently,  most  contracting  offices 
have  separate  support  offices  that 
perform  contracting  pricing. 
However,  providing  expert  pricing 
system  technology  at  the  individual 
contracting  officer/buyer  level  could 
fundamentally  alter  the  need  for  a 
separate  pricing  support  office. 
Expert  system  technology  has  the 
technology  to  allow  buyers  to  perform 
many  pricing  functions  by  themselves 
with  considerable  speed  and 
accuracy.  To  maximize  the 
advantage  of  expert  pricing  systems, 
buying  teams  might  be  used  that  do 
all  purchasing-related  functions,  to 
include  pricing.  If  a  separate  pricing 
office  was  to  be  maintained  then 
expert  systems  would  drive  up  the 
standards  for  productivity,  accuracy, 
and  timeliness  of  estimates. 


Expert  pricing  systems  could  also 
alter  the  composition  of  staffing  in 
contracting  offices.  Similar  to  other 
situations,  expert  systems  allow 
better  leveraging  of  organizational 
resources  by  permitting  a  shift  in  the 
workload  from  higher  to  lower  level 
experienced  personnel.  This 

leveraging  is  one  way  to  address  the 
recurring  problem  of  loss  of  expertise 
due  to  high  turnover  of  skilled 
contracting  personnel.  For 

contractors,  leveraging  means  being 
more  competitive  by  reducing  the 
cost  of  acquiring  subject-area 
expertise.  The  concept  of  leveraging 
is  summarized  below: 

The  aim  of  many  expert 
system  projects  is  to 
encapsulate  knowledge 
•out  a  circumscribed 
task  so  that,  instead  of 
junior  staff  bothering 
their  seniors  by  asking 
them  to  solve  routine 
problems,  the  juniors  can 
do  it  for  themselves  with 
the  aid  of  the  system. 

This  enhances  their 
sense  of  responsibility, 
allows  the  experts  to 
apply  themselves  to 
more  interesting 

problems,  and  may  result 
in  a  more  thorough  and 
more  consistent 

approach  to  the  problem 
than  the  frustrated 
experts  ever  provided. 

(16) 

The  expanding  use  of  expert  pricing 
systems  may  also  lead  to  concerns 
about  de-skilling  of  buyers/pricers 
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since  some  managers  and  personnel 
classifiers  may  believe  they  are 
redundant  compared  to  the  computer. 
To  address  this  issue,  it  is  important 
to  realize  expert  pricing  systems  are 
intended  to  assist,  not  replace,  their 
human  counterparts.  Expert  human 
analysis  and  evaluation  will  still  be 
required  in  many  cases.  However, 
the  productivity  of  those  performing 
the  price  and  cost  estimates  will 
improve  significantly  using  the  new 
technology. 

A  final  work  place  concern  is 
accountability  for  errors  or  omissions. 
Consider  the  issue  of  determining 
legal  liability  in  cases  where 

certification  of  cost  and  price  data  is 
required  by  Government  contractors. 
Or  in  case  of  the  Government,  if  a 
problem  such  as  major  cost  overrun 
occurs  when  using  an  expert  pricing 
system,  it  is  not  clear  who  should  be 
held  accountable,  e.g.,  the  cost/price 
analysts,  the  programmer,  the 

contracting  officer,  or  the  software 
vendor.  Care  should  be  taken  to 

ensure  expert  systems  are  not  used  to 
second  guess  human  cost/price 

analysts.  Contract  managers  should 
also  ensure  overreliance  on  the  expert 
system  does  not  occur.  Users  also 
need  to  understand  that  problems 
may  occur  if  they  attempt  to  use  the 
expert  pricing  system  outside  its 
range  of  knowledge  or  if  they  do  not 
account  for  geographical  variations  in 
the  prices  of  good  and  services. 

There  is  more  to  expert  pricing 
systems  than  just  automating  a  set  of 
rules  for  obtain  cost/price  estimates. 
There  are  significant  implementation 
issues  which  accompany  the 


introduction  of  this  new  technology 
into  the  contract  management 
workplace.  For  successful  use  of  this 
new  technology,  these  issues  must  be 
addressed  as  part  of  the  technical 
solution. 

Summary  and  Conclusions.  Artificial 
intelligence  is  a  new,  and  largely 
unharnessed,  technology  in  the 
contract  management  profession. 
However,  expert  pricing  systems  may 
be  a  primary  vehicle  for 
"mainstreaming"  the  use  of  Al  in  the 
contract  management  workplace. 
The  use  of  expert  system  technology 
will  become  more  widespread  as  the 
cost  of  microcomputer  hardware  is 
reduced.  Another  key  cost  incentive 
to  use  expert  systems  will  be  to  save 
money  by  better  leveraging  personnel 
resources  by  shifting  knowledge- 
related  work  from  higher  to  lower 
level  specialists;  this  is  an  important 
consideration  given  the  need  to  retain 
corporate  memory  in  critical  business 
areas/programs.  The  impact  of  cost 
will  continue  to  drive  the  use  of 
expert  systems  for  many  business 
applications  in  addition  to  the 
contract  pricing  function. 

As  a  way  to  improve  the  utility  of 
expert  pricing  systems  one  could 
reasonably  expect  innovations  such 
as  pre-prepared  modules  for  certain 
types  of  work,  e.g.,  professional 
services,  construction,  and  research 
and  development.  Another  useful 
improvement  would  be  the  use  of 
electronic  data  interchange  to 
automatically  keep  the  expert  pricing 
system  updated,  e.g.,  for  market 
changes  or  geographical  price 
variations.  Use  of  EDI  would  also 


297 


improve  accuracy  as  well  as  minimize 
buyer/price  workload.  One  key 
benefit  of  expert  systems  might  be  a 
leveling  of  the  playing  field  between 
large  and  small  businesses  when  it 
comes  to  pricing  proposals  and 
contract  modifications. 

Contract  managers  and  cost/price 
analysts,  in  addition  to  maintaining 
proficiency  with  the  FAR,  its 
implementing  directives,  and  industry 
trends,  will  also  be  required  to 
improve  their  automation  skills  to 
successfully  cope  in  the  21st  century 
contracting  office.  Expert  pricing 
systems  are  a  leading  application 
toward  the  realization  of  the  highly- 
automated  contracting  office  of  the 
future.  The  change  may  not  be  easy 
given  the  tradition  of  evolutionary 
acceptance  and  implementation  of 
new  automation  tools  in  the  contract 
management  work  place.  Managers 
and  users  need  to  realize  achieving  a 
payback  takes  time,  both  in  term  of 
learning  curve  and 

refinement/updating  of  the  expert 
system.  For  those  with  patience,  the 
sustained  long-term  rewards  of  using 
expert  pricing  system  technology  can 
far  exceed  the  initial  investment. 

Author's  Note:  Special  thanks  to  Mr. 
Dan  Walkovitz,  President  of  Mainstay 
Software  Corporation  in  Denver,  CO, 
who  served  as  a  reviewer  of  this 
research  paper. 

The  opinions  expressed  in  this  article 
are  solely  those  of  the  author. 
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Modeling  and  Simulation  for  Future  Bus  Architecture 
Patricia  Hoffman,  SPAWAR  421 


ABSTRACT 

The  scope  of  Modeling  and 
Simulation  has  broadened  significantly 
during  the  last  few  years.  Modeling  and 
Simulation  has  been  applied  to  many  areas  to 
which  it  was  never  before  associated.  This 
increase  in  applicability  of  Modeling  and 
Simulation  has  enabled  many  different 
groups  to  use  it  in  new  and  interesting  ways. 
One  of  those  ways  was  the  innovative  use  of 
simulations  to  evaluate  the  suitability  of  a 
communications  protocol  to  a  specified  Bus 
architecture.  The  topic  protocol  was  the 
IEEE  Futurebus+  protocol  which  was  under 
development  at  the  time  of  the  simulation. 
This  was  perhaps  the  first  time  that 
simulations  were  used  to  test  the  suitability 
of  a  draft  standard  for  a  particular 
architecture. 

INTRODUCTION 

The  scope  of  Modeling  and 
Simulation  has  broadened  significantly 
during  the  last  few  years.  Modeling  and 
Simulation  has  been  applied  to  many  areas  to 
which  it  was  never  before  associated.  This 
increase  in  applicability  of  Modeling  and 
Simulation  has  enabled  many  different 
groups  to  use  it  in  new  and  interesting  ways. 
One  of  those  ways  was  the  innovative  use  of 
simulations  to  evaluate  the  suitability  of  a 
communications  protocol  to  a  specified  Bus 
architecture.  The  topic  protocol  was  the 
Futurebus+  protocol  which  was  under 
development  at  the  time  of  the  simulation. 
The  method  of  evaluation  included 
simulation  as  no  hardware  existed  at  the  time 
of  the  study.  Futurebus+  was  still  in  the 
early  stages  of  development. 


This  paper  describes  the  modeling 
and  simulation  effort  which  supported  the 
evaluation  of  the  Futurebus+  protocol.  A 
brief  description  of  Futurebus+  will  be 
included,  followed  by  the  steps  taken  during 
the  course  of  this  evaluation,  and  then  a 
summary  of  some  of  the  lessons  learned 
during  this  project. 

FUTUREBUS+ 

The  Futurebus+  protocol  is  defined  in 
the  P896  series  IEEE  Protocol  Standards. 
Futurebus+  provides  the  means  for  the 
transfer  of  binary  information  between 
boards  over  one  or  more  logical  buses. 
Boards  may  contain  any  combination  of  one 
or  more  processors  and  local  resources  such 
as  cache,  memory,  peripheral  and 
communication  controllers,  etc.  The  logic 
protocol  is  defined  for  the  allocation  of  bus 
time  to  processors  which  need  tu 
communicate  with  other  modules  over  the 
bus.  The  Futurebus+  set  of  protocols  was 
designed  to  be  as  technology-independent  as 
possible  without  damaging  the  performance 
and  a  very  high  level  of  efficiency.  Many 
different  technologies  may  be  used  when 
implementing  the  bus  signals  (TTL, 
Backplane  Transceiver  Logic,  ECL,  CMOS, 
etc.)  as  long  as  the  Futurebus+  signaling 
conditions  are  met.j  Futurebus+  is  an  open 
architecture  protocol  with  features  such  as 
scalable  bit  width  (32/64/128  or  256  bits 
wide),  central  and  distributed  arbitration, 
cache  coherency,  and  live  insertion;  to  name 
a  few.  Any  implementation  that  meets  the 
Futurebus+  protocol  standard  is  a 
Futurebus+. 


301 


PROJECT  DESCRIPTION 

The  objective  of  this  project  was  to 
evaluate  the  performance  of  a  defined  bus 
architecture  that  used  Futurebus+  as  the 
communications  protocol.  The  goal  was  to 
examine  the  ability  of  this  bus  to  achieve  less 
than  10%  bandwidth  utilization. 

Before  going  into  the  actual 
modeling  and  simulation  of  Futurebus+,  a 
brief  description  of  the  tool  used  for  this 
project  will  follow.  This  will  enable  a  better 
understanding  of  the  specifics  of  the  model. 
The  tool  of  choice  for  the  modeling  and 
simulation  of  Futurebus+  was  OPNET™. 
OPNET™  is  a  Computer  Aided  Engineering 
(CAE)  tool  which  is  developed  by  MILS, 
Inc.,  a  firm  located  in  Washington,  D  C. 
OPNET™  is  an  event-driven  simulation  tool. 
Models  are  defined  using  a  hierarchical 
approach.  The  top  layer  is  the  Network 
Layer  which  defines  the  topology  or  layout 
of  the  architecture  to  be  modeled.  The  next 
layer  beneath  the  Network  Layer  is  the  Node 
Level.  The  Node  Level  describes  the 
functions  of  each  of  the  processors  defined  at 
the  Network  Level.  Several  types  of  nodes 
are  provided  in  OPNET™'s  extensive  library. 
These  include  a  fixed  communication  and 
mobile  communication  node  ^ype,  several 
predefined  data  generators,  and  several  types 
of  queue  nodes.  Beneath  the  Node  Level  is 
the  Process  Level.  The  Process  Level  is 
defined  using  state  diagrams  for  each  of  the 
nodes  defined  in  the  Node  Level.  Contained 
in  each  "bubble"  of  the  state  diagram  is  C 
code.  This  C  code  is  written  in  code 
fragments  and  the  tools  provided  with 
OPNET™  produce  a  complete  C  program 
based  on  all  information  contained 
throughout  the  three  outer  layers.  The  brief 
description  of  OPNET™  having  been 


completed,  the  actual  model  description  will 
follow. 

The  modeling  effort  was  completed 
in  6  phases  or  steps.  These  phases  were:  1) 
Definition  of  the  architecture  to  be  modeled, 
2)  Study  of  the  IEEE  Futurebus+  Draft 
Specifications  -  P896.1  (Logical  Layer 
Specification),  P896.2  (Physical  Layer 
Specification)  and  P896.3  -  (Recommended 
Practices),  3)  Design  and  Implementation  of 
the  Futurebus+  model,  4)  Data  Runs,  5) 
Analysis  of  Results,  and  6)  Present 
results/make  recommendations  to  the 
customer. 

Phase  1  began  with  the  definition  of  the 
topology  of  the  bus  to  be  modeled.  The 
architecture  to  be  modeled  was  defined  as  a 
communications  bus  with  8  processors 
logically  daisy-chained  together.  After 
obtaining  the  topology  of  the  architecture, 
the  performance  characteristics  of  each 
processor  were  defined.  Each  processor  was 
to  operate  at  a  different  data  rate.  Data  was 
to  be  transferred  between  nodes  in  packets 
and  packet  size  to  be  varied  for  each  node. 
The  total  available  bandwidth  was  800  Mbps 
given  the  specified  data  rates.  The 
arbitration  scheme  to  be  used  was  distributed 
arbitration. 

Distributed  arbitration  takes  place  in  six 
(6)  phases  or  steps.  Step  0  is  the  Idle  phase 
during  which  the  first  indication  that  a 
competition  is  beginning  takes  place.  The 
Decision  Phase  is  Step  1 .  During  this  phase 
a  processor  must  decide  whether  or  not  it 
wants  to  compete  for  bus  tenure.  The 
processor  must  assert  the  appropriate  signal 
in  order  to  compete.  During  Step  2  or  the 
Competition  Phase,  each  processor  monitors 
its  own  internal  win  signal  to  determine  if  it 
is  the  winner  of  the  arbitration.  The 
arbitration  process  is  a  priority  based  scheme 
with  the  highest  priority  processor  winning 
the  competition.  Upon  winning  the 
competition,  the  successful  processor 
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becomes  the  master  elect.  Stpp  3  is  the 
Error  Check  phase  which  is  used  to  ensure 
that  no  errors  occurred  during  the  previous 
competition  phase.  During  the  Master 
Release  phase  or  Step  4,  bus  tenure  is 
released  by  the  current  master.  In  Step  5, 
bus  tenure  is  transferred  to  the  winning 
processor.  The  master  elect  assumes  tenure 
of  the  bus.  During  Phase  1  of  the  project, 
the  draft  specifications  were  obtained  to 
enable  the  commencement  of  Phase  2. 

Phase  2  involved  studying  the  draft 
specifications  in  order  to  develop  a  working 
knowledge  of  the  Futurebus+  Protocol  and 
to  determine  which  of  its  features  were 
required  in  the  model  to  meet  the  defined 
goal  of  the  project.  During  phase  2,  it 
became  quite  apparent  that  the  scope  of  the 
modeling  effort  was  grossly  underestimated. 
Having  no  experience  with  the  family  of 
standards,  it  was  quite  difficult  to 
comprehend  the  protocol  specification.  Any 
underlying  assumptions  of  the  specification 
were  transparent  to  the  researcher.  The 
process  of  reading  and  understanding  the 
specifications  was  quite  laborious  and 
tedious.  After  developing  a  working 
knowledge  of  the  specification,  the  design  of 
the  simulation  was  begun. 

Phase  3  consisted  of  blending  the 
definition  of  the  architecture  to  be  modeled 
and  the  logic  defined  in  the  Logical  Layer 
specification  (P896.1  Draft  8.1)  into  the 
OPNET™  computer  model.  The  Futurebus+ 
model  was  designed  by  transferring  the 
defined  top  layer  topology  into  an  OPNET 
Network  model  which  consisted  of  a  bus 
with  8  nodes  or  processors  connected  to  the 
bus  via  bus  taps.  The  end  result  of  this  step 
is  depicted  in  Figure  1 .  Upon  completion  of 
the  OPNET  Network  model,  each  node  on 
the  Network  topology  was  defined  with  an 
OPNET  Node  model.  Figure  2  provides  an 
example  of  a  Node  Model  for  one  of  the 
processors. 


Each  of  the  Node  Models  contained  an 
Ideal  Packet  Generator,  which  generated 
data  at  defined  time  intervals  or  data  rates;  a 
transmitter;  a  receiver;  an  address  node,  a 
first-in-first-out  (FIFO)  queue;  and  the 
distributed  arbitration  node.  Upon 
completion  of  Node  model  definition,  each 
component  of  the  Node  model  was  defined 
by  an  OPNET  Process  model.  See  Figure  3 
for  a  sample  Process  Model.  The  Process 
model  defines  the  actual  performance  of  each 
of  the  nodes  on  the  bus. 

This  is  the  point  at  which  the  logic 
contained  in  the  Logical  Layer  protocol  was 
converted  to  C  code.  This  C  code  controlled 
the  transition  from  one  state  to  another  in  the 
Process  Model.  This  process  is  depicted  in 
Figure  4.  These  transitions  are  controlled  by 
scheduled  and  invoked  interrupts  which  are 
defined  within  each  process/state  via  the  C 
code. 

During  the  design,  several  ambiguities  in 
the  standard  were  discovered.  One  example 
was  the  definition  of  the  competition  number 
classes.  While  designing  the  implementation 
of  the  logic  flow  of  the  power-up  sequence, 
it  was  discovered  that  the  transition  from 
Power-up  to  Bus-Init  or  System_reset  or 
Bus  init  was  not  clear.  It  was  decided  to 
continue  the  design  assuming  that  the  system 
had  reached  a  steady  state  and  had  achieved 
power  up.  During  this  process,  the  IEEE 
Working  Group  for  Futurebus+  was 
contacted.  The  discussion  with  one  of  the 
lead  engineers  in  the  working  group  revealed 
that  the  standard  had  been  changed.  The 
current  draft  was  P896.1  Draft  8.2.  A  copy 
of  the  new  draft  standard  was  requested  and 
received  within  a  few  days.  Upon  receipt 
of  the  new  standard,  a  study  was  conducted 
to  evaluate  the  impact  on  the  design  to  date 
with  the  completion  of  any  required  changes. 
Upon  completion  of  Phase  3,  it  was  time  to 
move  on  the  debugging  and  testing  phase  of 
the  project. 
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Figure  2 
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Process  Level  Diagram 
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Phase  4  began  with  the  definition  of  the 
measures  of  effectiveness  to  be  used  to 
analyze  the  results  from  the  model.  The 
measures  of  effectiveness  used  for  the  model 
were  percent  (%)  bandwidth  utilized,  end-to- 
end  delay  time,  mean  delay  time  and 
throughput  rates.  The  next  step  to 
accomplish  was  the  definition  of  the  data  set 
to  be  used  for  the  evaluation.  The  first  data 
set  defined  the  data  rate  constant  at  each 
node  on  the  bus.  The  second  case  required 
the  generation  of  random,  uniformly 
distributed  data  rates  for  each  node  on  the 
bus.  Upon  completion  of  the  data  runs,  the 
results  were  analyzed. 

The  analysis  of  the  results  began  with 
the  computation  of  the  theoretical  maximum 
bandwidth  utilized  based  on  the  defined 
architecture.  This  calculation  was  based  on 
the  constant  data  rates  defined  at  each  node, 
the  size  of  the  packets  being  sent  by  each 
node,  and  the  bus  speed.  The  theoretical 
maximum  was  computed  including  the 
overhead  incurred  due  to  the  Futurebus+ 
protocol  and  with  no  overhead.  With 
protocol  overhead  considered,  the  calculated 
percent  bandwidth  utilized  was  2.68%.  The 
measured  bandwidth  utilization  from  the 
model,  which  included  protocol  overhead 
and  the  constant  data  rates,  was  2.6%. 
Upon  implementation  of  the  random  data 
rates,  the  measured  bandwidth  utilized 
remained  2.6%.  This  indicated  stability  in 
the  model.  The  measured  end-to-end  delay 
time  was  2.6  microseconds,  mean  delay  time 
was  430  nanoseconds  and  throughput  was 
20.7  Mbps.  These  results  are  demonstrated 
in  Figure  5.  Phase  5  ended  with  the 
conclusion  that  the  goal  of  less  than  10% 
bandwidth  utilization  could  be  achieved  for 
the  given  architecture  using  Futurebus+  with 
Distributed  Arbitration.  Upon  completion  of 
the  data  analysis.  Phase  6  began. 

During  Phase  6,  the  results  were 
presented  to  the  customer  along  with 


recommendations.  As  indicated  above,  the 
achievement  of  the  desired  goal  was 
successful.  Further  evaluation  was  requested 
to  determine  at  what  point  the  bus  would 
"break".  The  task  was  to  determine  the 
point  at  which  the  throughput  was  degraded. 
As  a  result,  further  test  cases  were  run. 
These  test  cases  were  designed  to 
incrementally  increase  the  data  rates  and  run 
the  model  with  each  set  of  parameters  until 
the  maximum  throughput  rate  was  achieved 
A  fixed  priority  scheme  was  used  for  the 
initial  runs.  These  results  appear  in  Figure  6. 
The  end-to-end  delay,  mean  delay,  and 
throughput  were  monitored.  The  results 
indicated  that  throughput  leveled  off  at  650 
Mbps.  Another  data  run  was  made  using  a 
random  priority  assignment  and  the  leveling 
off  point  occurred  at  650  Mbps. 

Throughout  the  course  of  this 
evaluation,  several  lessons  were  learned.  A 
brief  discussion  of  some  of  these  lessons 
follows. 


LESSONS  LEARNED 

The  lessons  learned  throughout  this 
process  were: 

1 .  Importance  of  Futurebus+ 

Expertise 

2.  Interaction  with  Working  Group 
Valuable 

3.  Required  flexibility  in  model  due 
to  dynamically  changing  draft 
standard. 


FUTUREBUS+  EXPERTISE 

When  this  project  first  began,  there  was 
no  engineering  support  available  with  a 
thorough  understanding  of  Futurebus+.  The 
process  of  learning  about  and  understanding 
the  specification  would  have  been  greatly 
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Measured  Throughput  (Mbps) 


improved  if  an  expert  Futurebus+  engineer 
had  been  available  to  ask  questions  and 
discuss  the  intent  behind  some  of  the 
sections  in  the  standard.  At  this  time,  the 
only  Futurebus+  experts  were  those 
engineers  involved  in  the  writing  of  the 
specification  and  they  were  not  available. 

INTERACTION  WITH  WORKING 
GROUP 

About  eight  (8)  months  into  the  project, 
we  were  asked  to  present  our  modeling 
efforts  at  one  of  the  IEEE  Futurebus+ 
Working  Group  sessions.  The  modeling 
project  was  presented  to  the  Fault  Tolerance 
group  and  the  Modeling  group.  It  was 
requested  that  we  present  a  status  report  at 
the  next  working  group  meeting  and  we 
agreed  to  provide  an  update  at  that  time. 
During  the  course  of  attending  the  working 
group  meetings,  a  discussion  with  one  of  the 
key  engineers  on  the  Working  Group 
revealed  an  underlying  assumption  which 
related  the  attribute  definition  and  the 
protocol  definition  contained  in  the  standard. 
This  valuable  insight  into  the  Logical  Layer 
specification  would  not  have  been  possible 
without  the  engineering  support  gained  at 
the  Working  Group  session. 

MODEL  FLEXIBILITY 

Another  lesson  learned  throughout  this 
project,  was  the  importance  of  flexibility  in 
the  model  when  working  with  a  draft 
standard.  This  became  apparent  when,  early 
in  the  modeling  process,  several  ambiguities 
were  discovered  in  the  specification.  One 
inconsistency  in  the  logic  protocol  was  the 
way  in  which  competition  numbers  were 
assigned  to  each  processor.  The  IEEE 
Futurebus+  Working  Group  was  contacted  . 
When  asked  about  the  table  contained  in  the 
logic  protocol,  it  was  determined  that  an 


outdated  version  of  the  draft  specification 
was  being  used.  An  updated  set  of 
Futurebus+  specifications  was  obtained.  The 
new  draft  standard  was  studied  to  evaluate 
the  impact  on  the  model.  After  determining 
changes  needed  in  the  model,  the 
modifications  were  completed.  Other 
discrepancies  in  the  standard  were 
discovered  during  the  debug  phase  of  the 
modeling  effort  and  it  was  discovered  that 
another  version  of  the  draft  standard  had 
been  released.  The  new  draft  standard  was 
obtained  and  studied  to  determine  the  impact 
on  the  model.  Required  changes  were  made 
and  the  debug  process  continued.  This 
process  took  place  several  times.  When  the 
project  began,  the  current  standard  was 
P896.1  Draft  8.3.  The  version  which  was 
approved  was  P896.1  Draft  8.5.  Building  the 
model  with  modularity  and  flexibility  was 
extremely  valuable  during  this  phase  of  the 
project. 

SUMMARY/CONCLUSIONS 

While  the  modeling  effort  involved  a 
very  small  portion  of  the  family  of 
Futurebus+  specifications,  the  usefulness  of 
modeling  and  simulation  in  the  development 
of  a  protocol  standard  was  demonstrated. 
Even  though  many  of  the  ambiguities  and/or 
errors  were  corrected  in  the  revised 
standards,  the  specification  of  the  logic 
protocol  in  a  formal  language  would  enable 
the  detection  of  errors  and  ambiguities 
quickly  and  would  be  cost  effective.  The 
engineers  developing  the  standard  could 
work  hand  in  hand  with  the  software 
engineers  to  insure  proper  implementation  of 
the  logic.  Working  together  these  engineers 
would  form  a  team  to  evaluate  the  current 
design  and  study  possible  improvements. 

The  usefulness  of  modeling  and 
simulation  was  also  demonstrated  in  the 
testing  of  a  proposed  bus  architecture  built 
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on  a  specified  logic  protocol.  The  use  of 
hardware  was  not  possible  as  none  existed  at 
the  time  this  project  initially  started. 
Modeling  and  simulation  could  be  used  to 
test  the  performance  of  the  bus  architecture 
using  any  number  of  communication 
protocols.  These  results  could  be  compared 
to  determine  the  best  choice  for  the  task  at 
hand. 

The  field  of  modeling  and  simulation  has 
indeed  changed  over  the  last  few  years.  Its 
recent  growth  has  enabled  groups  of  all  types 
to  use  it  in  new  and  interesting  ways.  One  of 
those  ways  was  the  innovative  use  of 
simulations  to  evaluate  the  suitability  of  a 
communications  protocol  to  a  specified 
Communications  Bus  architecture. 
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EVOLUTIONARY  ACQUISITION 


of 

Command  and  Control  Systems 
in  Practice: 

The  Operations  Support  System 


ABSTRACT 

In  the  face  of  rapidly  changing 
computer  technology  (hardware  and 
software),  the  use  of  conventional 
approaches  to  command  and  control  (C^) 
system  acquisition  has  often  produced 
unsatisfactory  results.  Evolutionary 
Acquisition  (EA)  is  an  innovative 
acquisition  strategy  being  utilized  by  the 
Space  and  Naval  Warfare  Systems 
Command's  Command  Systems 
Directorate  to  acquire  software  intensive, 
highly  interactive  C^  systems. 

This  paper  describes  the 
implementation  of  the  EA  approach  on 
one  such  system,  the  Operations  Support 
System  (OSS).  OSS  is  a  key  element  of 
the  Navy  Command  and  Control  System 
(NCCS)  which  is  being  developed  and 
implemented  by  the  NCCS  Program 
Office.  This  paper  examines  EA  lessons 
learned  by  the  Program  Manager  and  the 
OSS  Project  Manager.  It  describes  the 
character  of  EA  as  implemented  and  the 
organizational,  political,  and  human 
aspects  of  acquiring  a  system  using  the 
EA  strategy. 


The  decision  to  acquire  OSS  using 
EA  has  proved  correct.  However,  there 
were,  and  are  still,  difficulties.  The  "Joint 
Logistics  Commanders  Guidance  for  the 
Use  of  an  Evolutionary  Acquisition  (EA) 
Strategy  in  Acquiring  Command  and 
Control  Systems"  provided  cautions 
designed  to  alert  acquisition  leaders  to  the 
necessity  for  "proper  implementation"  of 
EA.  Two  are  quoted  here:  "Successful 
execution  of  an  EA  program  requires  a 
number  of  changed  relationships  and 
practices  common  to  more  conventional 
acquisition  programs....  Establishing 
effective  patterns  of  interaction  with 
external  organizations  involved  in  an 
evolutionary  acquisition  can  be  expected 
to  be  unusually  difficult,  because  the  very 
nature  of  EA  requires  relationships  and 
interactions  different  from  the  norm." 

The  EA  concept  is  being 
successfully  used  to  acquire  the  OSS  by 
SPAWARSYSCOM. 
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IMTRODUCTIQN 

Evolutionary  acquisition  (EA)  is  a 
singularly  effective  approach  to  acquiring 
command  and  control  systems.  EA  applies 
best  to  programs  for  which  conventional 
acquisition  strategies  will  lead  to 
unsatisfactory  results  (e.g.,  the  multi-year 
development  and  subsequent  delivery  of  an 
already  obsolete  system  because  rapidly 
advancing  technology  outdistanced  the 
original  system  requirements  and 
specifications  during  the  acquisition  process). 
This  makes  EA  particularly  well  suited  for 
command,  control,  communications, 
computers  and  intelligence  (C4I)  systems, 
where  technology,  component  performance 
and  operational  necessity  are  a  continuously 
moving  target.  Other  Navy  C4I  acquisitions 
have  now  transitioned  from  unique,  turn-key 
developments  to  EA,  in  particular  the  Navy 
Tactical  Command  System  Afloat 
(NTCS-A),  the  OBU  Evolutionary 
Development  (OED),  and  the  ASW 
Operations  Center  (ASWOC)  Upgrade 
programs.  But  there  has  been  only  one 
acquisition  effort  designed  from  its  inception 
to  invoke  an  EA  strategy  (EAS).  This 
development,  the  OSS,  has  successfully 
broken  new  ground  in  its  approach  to 
systems  acquisition,  and,  as  directed  in  1988 
by  the  Assistant  Secretary  of  the  Navy, 
serves  as  the  Navy's  model  for  evolutionary 
acquisition  strategy.  The  successes  of  this 
model  Navy  program  have  served  as  the 
impetus  for  other-service  and  joint  program 
efforts  to  join  the  EA  revolution. 


EVOLUTIONARY  ACQUISITION 
STRATEGY  DEFINITION 

EA  was  originally  publicized  in 
articles  and  studies  in  the  1980s  and 
championed  by  BGen  Edward  Hirsch,  USA 
(Ret.)  of  the  Defense  Systems  Management 


College  (DSMC).  However,  it  is  still 
modestly  documented  in  the  Department  of 
Defense.  DoD  instruction  5000.2  states: 
"Alternative  acquisition  strategies  should  be 
considered  for  systems  where  requirements 
refinements  are  anticipated  or  where  a 
technology  risk  or  opportunity  discourages 
immediate  implementation  of  a  required 
capability.  Alternative  acquisition  strategies 

include  evolutionary  acquisition . 

Evolutionary  acquisition  is  an  approach  in 
which  a  core  capability  is  fielded,  and  the 
system  design  has  a  modular  structure  and 
provisions  for  future  upgrades  and  changes 
as  requirements  are  refined.  An  evolutionary 
acquisition  strategy  is  well  suited  to  high 
teclinology  and  software  intensive  programs 
where  requirements  beyond  a  core  capability 
can  generally,  but  not  specifically,  be 
defined.  This  approach  is  described  in  Joint 
Logistics  Commanders  Guidance 
'Evolutionary  Acquisition,  An  Alternative 
Strategy  for  Acquiring  Command  and 
Control  (C^)  Systems'."' 

The  JLC  Guidance  succinctly  defines 
EA  as  "an  acquisition  strategy  which  may  be 
used  to  procure  a  system  expected  to  evolve 
during  development  within  an  approved 
architectural  framework  to  achieve  an  overall 
system  capability.  Underlying  factors  in  EA 
include: 

•  The  need  to  field  a  well  defined  core 
capability  quickly  in  response  to  a 
validated  requirement,  while  planning 
through  an  incremental  upgrade  program 
to  eventually  enhance  the  system  to 
provide  the  overall  system  capability. 

•  One  or  more  increments  (comprising  the 
incremental  upgrade  program),  which  are 
treated  as  individual  acquisitions.  The 
increments'  scope  and  content  are  the 
result  of: 
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-continuous  feedback  from  developing 
and  independent  testing  agencies,  the 
user  (operating  forces),  and  supporting 
organizations 

-the  desired  application  of  new 
technology  balanced  against  the 
constraints  of  time,  requirements  and 
cost. 

As  defined  above,  evolutionary 

acquisition  comprises  the  following 

elements; 

•  A  concise  statement  of  operational 
concepts  and  requirements  for  the  full 
system. 

•  A  general  description  of  the  functional 
capability  desired  for  the  full  system. 

•  A  flexible,  well  planned  overall 
architecture,  to  include  process  for 
change,  which  will  allow  the  system  to 
be  designed  and  implemented  in  an 
incremental  way. 

•  A  plan  for  incremental  achievement  of 
the  desired  total  capability. 

•  Early  definition,  funding,  development, 
testing,  fielding,  supporting,  and 
operational  evaluation  of  an  initial 
increment  of  operational  capability. 

•  Sequential  definition,  funding, 
development,  testing,  fielding,  supporting 
and  operational  evaluation  of  additional 
increments  of  operational  capability. 

•  Continual  dialog  and  feedback  among 
users,  developers,  supporters  and 
testers."^ 


OSS’s  EA  BACKGROUND 

OSS  evolved  in  the  mid  1980s  from  user 
discontent  over  the  traditional  approach  to 
program  development,  in  particular  for  FCC 
support  systems.  Fleet  input  eventually 
resulted  in  a  Navy  Warfare  Requirements 
Board  review  and  VCNO  direction  that  a 
new  program  be  initiated  with  an  FY  90 
start.  Direction  also  provided  for  a  new 
approach  in  acquiring  this  system.  CNO 
promulgated  an  OSS  Operational 
Requirement  in  December  1987.  After  many 
behind  the  scenes  fact  finding  and  operations 
requirements/needs  discussions  among  CNO 
and  fleet  representatives,  the  OSS  Project 
Manager,  and  ASN  (RE&S),  and  finally  a 
Program  Review  in  September  1988,  OSS 
was  approved  as  an  "Evolutionary 
Acquisition  Strategy"  (EAS)  program  by 
ASN  (RE&S)  17  October  1988. 

A  caveat  to  this  approval  was  the 
requirement  that  "an  assessment  of  the 
planned  OSS  will  be  performed  by  an 
independent  agent  under  ASN  (RE&S) 
sponsorship  prior  to  any  final  decision  on 
release  of  OPN  procurement  authority". 
This  study  was  conducted  by  a  select 
committee  of  six  experts  in  Navy  command 
and  control,  OSS  related  technologies, 
systems  development,  and  DoD  acquisition 
processes.  Noteworthy  was  the  presence  of 
BGen  Hirsch  among  the  committee 
members.  The  committee  strongly  supported 
the  OSS  acquisition  strategy  and 
recommended  that  ASN  (RE&S)  approve  the 
proposed  OSS  approach.  They  added,  ".... 
Candidly  stated,  the  Assessment  Study 
Group  feels  that:  The  Assistant  Secretary  of 
the  Navy  (Research,  Engineering  and 
Systems),  as  well  as  the  Program  Sponsor 
(OP-094),  COMSPAWARSYSCOM  and  the 
Program  Director  must  be  active  supporters 
of  the  Program  Management  team's  efforts  to 
implement  successfully  the  EA  approach  to 


313 


acquiring  the  OSS.  The  above  named 
leaders  should  cause  their  staffs  to  be 
favorably  responsive  to  the  program  team’s 
efforts  to  tailor  the  conventional  acquisition 
process  to  accord  with  EA  principles. 
Specifically,  granting  an  exception  to  the 
logistics  community  to  require  a  Logistics 
Review  Group  audit  —  albeit  "modified"  — 
appears  to  be  a  manifestation  of  a  rigid  mind 
set  by  our  bureaucracy  that  can  frustrate 
attempts  by  the  management  team  to 
accomplish  its  mission".^ 

Following  this  recommendation  (and 
prophetic  advice),  ASN  RE&S  completed  the 
program  approval  process,  reviewed  the 
program  again  in  an  October  1989  Navy 
Program  Decision  Meeting  (NPDM),  and 
provided  authority  for  the  first  two 
Increments  of  OSS  to  proceed.  Funding  for 
OSS  as  a  separate,  approved  program  began, 
on  schedule,  in  FY  90.  Establishing  its 
precedent  for  quickly  providing  capability  to 
the  end  user,  OSS  was  in  the  process  of  its 
first  operational  hardware  installation  less 
than  two  months  later. 


OSS  SYSTEM  DESCRIPTION 

OSS  is  an  integral  element  of  the 
Navy  Command  and  Control  System 
(NCCS)  Ashore  Program,  a  network  of 
command  and  control  components  directly 
supporting  commanders  and  their  watch 
teams,  including  echelons  from  the  Chief  of 
Naval  Operations  (CNO)  through  the  Fleet 
Command  Centers.  As  a  result  of  its  wide 
acceptance,  OSS  has  also  b^en  installed  in 
the  Unified  Command  Centers,  NATO 
Command  Centers,  Foreign  Navy  Command 
Centers,  MSC  Headquarters,  and  will  be  the 
core  of  the  PACOM  Crisis  Management 
System.  Additionally,  it  is  installed  at 
SACLANT  and  CINCIBERLANT  and  is 
being  considered  for  diverse  NATO  and 


other  foreign  command  system 
developments.  OSS  is  a  critical  component 
of  the  combined  Command  and  Control 
thrust  of  the  Navy  Command  Systems  (NCS) 
consolidation  effort  currently  ongoing  in 
SPAWARSYCOM.  This  is  essentially  an 
effort  to  provide  an  optimal  system  of 
systems  to  support  the  afloat  and  ashore 
Navy  C2  communities  and  their  interfaces 
with  joint  and  foreign  systems. 

OSS  was  initially  planned  as  a  means 
of  replacing  the  functionality  of  three 
primary  prototypes  and  one  aging  C2  system, 
but  has  evolved  to  include  far  more 
functionality.  It  is  itself  a  system  of 
systems,  and  employs  extensive  use  of 
accepted  government  and  commercial 
standards,  and  varying  amounts  of 
Commercial-off-the-Shelf  (COTS)  and 
Govemment-off-the-Shelf  (GOTS)  software. 
It  also  exploits  the  use  of 
Non- Developmental  Item  (NDI)  and  COTS 
hardware  to  the  maximum  extent.  Because 
of  the  rapid  and  progressive  increases  in 
NDI  technology  and  performance  at  a  low 
investment,  OSS  can  optimize  functionality 
and  performance  on  a  continuous  basis. 
Savings  also  accrue  through  the  lack  of 
dependence  on  a  long  logistics  trail.  OSS  is 
a  distributed  processing  system,  allowing 
software  functions  and  components  to  be 
distributed  among  workstations  and  other 
computing  resources  to  achieve  optimum 
system  p>erfonnance  within  a  given 
configuration.  It  employs  a  minimum  core 
element  to  provide  charting,  tactical  display 
and  various  housekeeping  functions,  and  a 
modular  structure  for  the  rest  of  the  system 
permitting  addition,  deletion  and 
modification  of  functions  as  the  need  arises. 
Definition  of  inter-module  interfaces  is  via 
Application  Programmer  Interfaces  (API's). 

Initial  OSS  workstation 
configurations  consisted  of  Navy  standard 
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Desktop  Tactical  Computers  (DTC-2)  based 
on  the  SUN  Reduced  Instruction  Set 
Computer  (RISC)  architecture  and  the  UNIX 
operating  system.  Current  development 
employs  the  Hewlett-Packard  TAC-3  Navy 
standard  computers  employing  the  HP  OS 
also  operating  in  a  Unix  environment.  As  a 
point  of  interest,  the  EA  strategy  allowed  the 
Helding  of  the  leading  edge  technology 
TAC-3  in  a  matter  of  weeks  after  contract 
award.  This  type  response  is  nonexistent  in 
traditional  acquisition.  Workstations 
communicate  over  a  Local  Area  Network 
(LAN)  using  the  Transmission  Control 
Protocol/Intemet  Protocol  (TCP/IP).  Typical 
workstation  functions  can  be  economically 
achieved  through  devices  such  as  X 
terminals  or  multiple  remote  monitors  and 
keyboards  off  the  powerful  TAC-3.  OSS 
extensively  employs  widely  used  standards 
including  the  emerging  Common  Operating 
Environment  for  Command  and  Control 
Systems,  X-windows,  Motif,  UNIX,  C, 
ADA,  an  IEEE  802.3  network  with  TCP/IP 
protocol,  database  access  via  SQL,  extensive 
use  of  applications  interface  standards,  and 
the  Navy  standard  computers.  Significant 
COTS  products  include  Oracle,  Motif, 
LMDS,  Textract,  and  the  Office  Automation 
System,  ASTER*X. 

OSS  functions  are  centered  around  an 
extensive  capability  to  receive,  process, 
display,  and  assess  the  readiness  and 
disposition  of  own,  neutral,  and  potentially 
hostile  forces.  Specifically,  these  functions 
include: 

•  An  integrated  database  providing 
characteristics,  performance  and  status  of 
forces  information,  tactical  data,  and 
record  messages  with  both  automatic  and 
user  controlled  updating 

•  Data  retrieval  through  user  tailorable 
forms,  and  automated  query  and  report 


generation 

•  Tactical  display  utilizing  track  positional 
information.  Multi-projection, 
Multi-resolution  maps.  Multi-source 
maps,  predefined  geographic  areas, 
overlays  to  maps,  tactical  decision  aids, 
and  Near  real-time  update  of  positional 
data,  and  the  ability  to  save,  display,  and 
print  screens 

•  Fixed  location  database  and  display 
capability  for  ports,  airfields  and  key 
assets 

•  Communications  including  AUTODIN 
connectivity,  TTY,  OTCIXS  and  Tactical 
Link 

•  Message  handling  and  routing  using 
extensive  profiler  and  retrospective 
search  capabilities 

•  Message  Processing  including  creation 
validation  (222  MTF  Types),  Input  and 
Output  message  review,  and  message 
error  correction 

•  Office  Automation  including  Desktop 
Publishing,  Spread-sheet,  Graphics,  Word 
Processing  and  E-mail 

•  Briefing  support  with  Large  Screen 
Displays,  Video  Switching,  and  various 
site  selected  monitors 

Functions  to  be  provided  in  near  tenn 

releases  include 

•  Joint  Sorts  information  database  and 
display 

•  Additional  message  format  processing 

•  Improved  message  error  correction 
capability 
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A  typical  OSS  Increment  I  suite  is 

•  Exercise  database  capability  shown  below. 

•  Employment  scheduling  integration 

•  Query  Assist  Language  improvements 

•  Additional  screen  generation  and  tote 
boards  capability 

•  Consolidated  history  file 

•  Enhanced  Message  handling  capabilities 

OSS  is  being  developed  in  three  increments, 
with  the  first  providing  capability  by  having 
systems  working  with  each  other  over  a 
Local  Area  Net.  The  second  increment 
provides  a  system  of  functions  operating 
over  the  net,  with  the  third  providing 
enhancements  and  increases  in  functionality 
to  the  second  increment. 


As  the  system  matures,  the  individual  performing  as  modular  functions  on  a  net. 
systems  will  lose  their  identity  in  favor  of 
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The  following  diagram  shows  how  OSS  will 
look  as  a  fully  integrated  system . 


This  latter  system  concept  fully  embodies  the 
EA  approach.  It  creates  the  requisite 
framework  wherein  functions  can  be 
augmented  or  replaced  with  minimal  impact 
on  the  rest  of  the  system,  and  at  minimal 
expense.  It  allows  not  only  increased 
responsiveness  in  providing  additional 
functionality  based  on  technology  and/or 
operational  need,  but  also  the  capability  to 
mix  and  match  functions  to  meet  the  varying 
needs  of  different  end  users. 


EVOLUTIONARY  ACQUISITION  IN 
PRACTICE . . IS  IT  WORKING? 


Even  with  the  apparent  successes  of 
OSS  the  question  is  often  asked  "is  the  EA 
concept  working?"  In  short,  the  answer  for 


OSS  is  an  unqualified  yes;  it  has  worked 
extremely  well.  It  has  provided  capability 
far  beyond  that  envisioned  in  the  planning 
for  similar  but  monolithic  developments  in 
the  mid  80's,  at  a  fraction  of  the  cost.  It  has 
also  provided  a  proving  ground  for  new  and 
more  efficient  methods  in  procurement,  such 
as  implementation  of  Navy  standard 
computers,  establishment  of  standards  which 
serve  multi-service  needs,  and  development 
of  a  common  opierating  environment  that  will 
have  world  wide  impact. 

Of  particular  importance  and  success 
since  the  early  stages  of  OSS  was  the 
development  of  the  EA  unique  relationships 
with  the  user.  Because  this  community's 
involvement  initially  assisted  in  the 
definition  of  OSS  as  an  EA  program,  the 
relationship  has  progressed  smoothly  and 
quickly.  The  users  are  now  an  integral  part 
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of  the  OSS  teain  from  the  earliest  stages  of 
each  development  through  the  delivery, 
operational  use,  and  maintenance  of  the 
system.  Tliis  "team"  concept  is  manifested 
in  rniiny  user/developer  dialogues,  most 
significantly,  the  Fleet  Project  Team  (FPT), 
Teclmical  Working  Group  (TWG),  and  the 
NCCS  Change  Request  (NCR)  process. 

The  OSS  FPT  was  established  to 
enhance  coordination  among  the  sponsor, 
users,  and  developers  of  the  system.  It  is 
convened  at  least  semiannually  to  review  the 
status  of  incremental  developments  with  the 
users  and  to  provide  high  level  user  feedback 
to  OPNAV  and  SPAWARSYSCOM 
concerning  system  performance  and 
supportability.  In  addition,  the  FPT  provides 
the  forum  for  identifying  and  prioritizing 
emergent  requirements,  and  discussing 
development  options.  The  FPT  is  chaired  by 
the  OSS  Resource  Sponsor  and  is  comprised 
of  Fleet  and  PMO  representatives. 

TWG  meetings  are  also  conducted  at 
least  semiannually,  usually  preceding  FPT 
meetings.  The  purpose  of  the  TWG 
meetings  is  to  provide  a  technical  forum  to 
detemiine  the  status  of  incremental  system 
developments,  identify  problems  at  user  sites 
and  to  promulgate  guidance  to  engineering 
activities  and  contractors.  TWGs  are  chaired 
by  the  OSS  Project  Manager  with 
participation  from  the  Resource  Sponsor's 
office.  Project  Management  Organization 
(PMO),  engineering  activities,  development, 
integration  and  support  contractors,  and 
users. 

The  NCR  is  the  tool  used  to  provide 
a  common  means  for  communicatmg 
information  and  priorities  among  the  various 
participants  in  the  OSS  program.  NCRs  may 
be  submitted  by  any  OSS  program 
participant  to  document  a  request  for  change, 
new  requirements  or  problems  encountered 


in  OSS  operation  which  require  correction. 
NCRs  are  the  primary  tool  in  the  change 
process  and  are  the  common  ground  for 
dialogue  in  the  various  OSS  forums  (e.g., 
FPT,  TWG,  CCB).  Though  often  thought  of 
as  unglamorous,  this  is  perhaps  the  most 
powerful  and  dynamic  communication  tool 
for  ensuring  the  evolving  system  meets  the 
end  users'  needs  and  is  properly  documented. 

These  forums  and  the  day-to-day 
interaction  between  the  Project  Management 
team,  the  OSS  users  and  developers  have 
ensured  that  a  team  concept  exists. 

Another  key  area  in  the  evolutionary 
acquisition  of  OSS  is  participation  of  the 
Commander,  Operational  Test  and 
Evaluation  Force  (COMOPTEVFOR). 
OPTEVFOR  is  the  key  player  in  the  testing 
of  OSS.  In  EA,  one  of  the  changed 
relationships,  and  certainly  one  of  the  most 
important,  is  the  relationship  between  the 
developer  and  the  the  test  community,  as 
epitomized  by  OPTEVFOR,  the  independent 
tester.  The  JLC  Guidance  says;  "the 
independent  tester  is  an  important  player  in 
this  process.  It  is  imperative  that  he  become 
involved  early  in  the  program  development 
phase  and  maintain  a  direct  and  continuous 
liaison  with  the  developer  and  user 
throughout  the  EA  process,  so  that 
operational  test  and  evaluation  can  proceed 
with  maximum  rapidity".  The  frequency  of 
program  incremental  deliveries  necessitates 
similarly  rapid  developmental  and 
operational  testing.  This  clearly  demands  a 
changed  relationship  with  the  op)erational  test 
community  (OPTEVFOR). 

OPTEVFOR  and  the  OSS  Project 
Manager  have  successfully  established  this 
relationship.  From  the  earliest  stages  of  OSS 
development,  OPTEVFOR  has  been 
involved.  OPTEVFOR  provides  a 
representative  to  the  FPT  and  the  OSS 
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Configuration  Control  Board  (CCB). 
Additionally,  the  OSS  Project  Manager 
requests  OPTEVFOR  representation  at  every 
test  stage  of  development/acquisition,  e.g., 
OPTEVFOR  is  requested  to  monitor 
developmental  and  integration  testing  at 
contractor  and  government  sites  in  addition 
to  their  chartered  responsibility  to  conduct 
operational  testing. 

Though  SPAWARSYSCOM  and 
OPTEVFOR  have  established  a  successful 
relationship,  this  area  is  also  not  without 
problem.  The  identification  of  test 
thresholds,  the  problem  in  identification  of 
testable  functions  within  the  respective 
hardware  and  software  releases, 
determination  of  what  constitutes  a  testable 
release  under  an  EA  environment,  and 
how/when  functionality  should  be  released  to 
the  fleet  are  sill  undergoing  definition.  More 
definitive  documented  policy  for  EA  and 
COTS/NDI  efforts  would  be  most  helpful. 
There  is  an  effort  underway  to  do  this.  The 
arbitrary  and  artificial  requirement  to  update 
the  Test  and  Evaluation  Master  Plan  (TEMP) 
for  the  sake  of  update  when  no  major 
programmatic  or  functional  change  has  taken 
place  is  not  cost  effective. 

An  integral  component  of  user  and 
test  community  involvement  is  the 
innovative  and  vital  program  of  Beta  testing 
(BT),  which  was  established  at  user  sites. 
Beta  testing  is  an  evolutionary  test  concept 
adopted  by  the  OSS  Program  to  involve 
command  center  users  in  the  development 
effort  ensuring  system  operability  prior  to 
each  general  release  of  OSS.  One  command 
center  site  is  selected  as  the  OSS  "Beta  Site" 
for  each  release  following  Developmental 
Testing  (DT)  in  the  lab.  BT  offers  two 
primary  advantages  over  Developmental 
Testing  at  a  laboratory.  First,  an  actual 
command  center  involves  live  message 
traffic  and  communications.  Second, 


command  center  personnel  offer  a  more 
realistic  measure  of  system  operability  and 
suitability  than  test  engineers.  OPTEVFOR 
monitoring  of  Beta  Testing  at  a  live 
command  center  helps  ensure  that 
operational  testing  will  be  expedient  and 
successful. 

While  more  time  and  several  test 
cycles  and  deliveries  should  be  completed 
before  a  verdict  regarding  this  aspect  of 
OSS/EA  implementation  can  be  reached,  it  is 
clear  that  a  powerful  foundation  has  been 
established.  It  is  also  clear  that  this  progress 
is  chiefly  a  result  of  both  the 
COMOPTEVFOR  and  OSS  Project 
Manager's  unbiased  attempts  to  avoid 
inflexible  mindsets  in  this  critical  area. 
Changes  to  directives,  regulations  and 
policies  are  required  to  institutionalize  this 
process. 

One  of  the  significant  remaining 
actions  necessary  to  unlock  the  potential  of 
Evolutionary  Acquisition  is  the  very  one 
initially  identified  early  in  program 
development,  and  stressed  during  the 
Independent  Assessment.  That  aspect  which 
has  not  been  streamlined  is  the  programmatic 
and  logistics  review  process,  which  have 
been  somewhat  inflexible  and,  in  the 
extreme,  not  productive  or  cost  effective. 
Most  other  reviews,  documentation  and 
development  approaches  have  successfully 
been  modified  to  meet  an  EA  program's 
needs,  resulting  in  a  streamlined  approach  to 
the  acquisition.  But  as  warned  during  the 
independent  assessment,  the  logistics  review 
process  has  changed  little.  The 
programmatic  time  and  manpower  put  into 
responding  to  program  investigations  of 
questionable  value  has  resulted  in  potential 
development  dollars  going  toward 
"bureaucratic"  requirements.  Findings  of 
these  audits  have  shown  that  the  inflexibility 
of  the  directives,  regulations  and  policy  can 
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be  detrimental  to  the  evolution  of  a 
successful  program.  Unrealistic  requirements 
to  create  supply  lines  that  will  take  three 
years  to  put  in  place  for  hardware  that  will 
be  in  place  only  three  years  at  the  most, 
personnel  requirements  for  unnecessary 
positions,  and  rigid  waiver  requirements  for 
Commercial  off  the  Shelf  products  for  ashore 
systems  that  have  no  need  to  meet  the  same 
rigid  standards  as  for  afloat  use  are  but  a 
few  examples  of  standard  acquisition 
program  requirements  inflexibly  forced  on 
the  OSS  program.  These  requirements  are 
not  consistent  with  the  concept  of  EA  or  the 
acquisition  of  C4I  systems.  Further,  the 
resources  allocated  to  meet  these  extraneous 
requirements  are  inevitably  diverted  from  the 
acquisition  of  hardware  and  software.  Until 
changes  are  made  in  the  development 
oversight  process,  there  will  be  a  thorn  in  the 
side  of  Evolutionary  Acquisition. 

Additionally,  the  program  review 
process  has  yet  to  completely  come  to  grips 
with  EA.  When  should  a  program  be 
reviewed?  What  requires  Navy  Acquisition 
Executive  approval?  With  frequent  software 
releases,  some  combined  with  hardware 
upgrades,  and  numerous  increments  and 
phases,  what  constitutes  Initial  Operational 
Capability,  Full  Operational  Capability,  etc? 
These  are  but  a  few  of  the  dilemmas 
routinely  faced.  Thus  far,  dedicated  people 
with  open  minds  have  worked  these  issues 
for  the  OSS,  but  long  term  the  documented 
process  must  be  changed.  As  the  acquisition 
work  force  down  sizes  and  program  funds 
are  reduced,  efficiency  and  streamlining  are 
mandatory. 

Regardless,  and  in  spite  of  the 
problems  with  acquisition  regulations  and 
policy,  OSS  as  an  EA  has  worked  extremely 
well  and  has  been  exceptionally  successful. 
The  many  successes  of  OSS  have  proven  the 
concept  works; 


•  Rapid  deployment  of  capability  has  been 
proven.  The  installation  of  the  first  OSS 
computers  shortly  after  the  program  first 
started  followed  by  frequent  software 
releases  providing  increased  functionality 
and  tailoring  and  fine-tuning  of  existing 
functions  has  demonstrated  this. 

•  Capability  to  meet  fleet  requirements  on 
a  set,  realistic  and  frugal  budget  has  been 
proven.  Significant  capability  has  been 
provided  at  a  fraction  of  the  cost  of  that 
initially  envisioned  for  other  late  80's 
system  developments  of  this  magnitude. 

•  It  has  adequately  demonstrated  that  a 
program  can  be  flexible  enough  to  bend 
with  a  changing  budget  envirorunent 
without  toppling  the  system  development 
or  ending  in  significant  system  delivery 
delays.  Decreases  in  budget  are  met 
with  a  reduction  in  delivered 
functionality,  rather  than  a  significant 
delay  in  the  total  system  delivery. 
Increases  in  budget  provide  the  bonus  of 
additional  functionality  or  systems 
upgrades  that  can  generally  be  realized  in 
the  same  budget  year 

•  It  has  proven  that  with  EA,  a  build  a 
little,  test  a  little,  field  a  little  approach 
will  provide  a  reliable  and  extremely 
capable  system  to  the  end  user.  It  also 
allows  fixes  and  tailoring  to  be  done  in 
parallel  with  the  development  of  new 
functionality. 

•  The  development  approach  has  shown 
that  creation  of  and  reliance  on 
mandatory  standards  for  software 
development,  hardware  and  interfaces  is 
invaluable  to  EA.  These  standards  are 
the  keystone  on  which  all  developments 
and  future  changes  and  enhancements 
can  evolve.  Development  of  a  formally 
defined  common  operating  environment 
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was  also  key  to  ensuring  other  systems 
and  components  developed  in  parallel  are 
compatible  and  has  afforded  the 
leveraging  of  other  investments  and  the 
synergy  of  effort. 

As  the  OSS  system  matures,  it  has 
and  will  provide  a  wealth  of  knowledge 
based  on  first  hand  experience  in 
evolutionary  development.  Even  with  the 
successes,  the  concept  and  the  program  are 
not  without  fault.  The  overall  development  is 
subject  to  the  personalities  of  the  Sponsor, 
the  Project  Manager,  the  System 
Integrator/Engineer  and  Developers,  and  the 
Users.  Time  will  be  the  judge  of  the  success 
of  the  OSS  program  strategy.  It  was  created 
by  its  initiators  in  the  belief  that  the  concept 
could  work.  These  pioneers  of  the  EA 
strategy  carved  a  plan  which  was  responsive 
to  the  user,  met  their  requirements,  was 
reliable  and  quickly  fielded,  and  which  could 
exist  in  a  rapidly  changing  funding 
environment.  Full  program  management 
commitment  and  dedication  to  the  concept 
from  the  top  down  combined  with  further 
streamlining  of  the  acquisition  process  will 
ensure  the  EA  development  concept  will 
continue  to  succeed. 

The  EA  strategy  for  OSS  has  achieved 
these  success  in  spite  of  a  natural  resistance 
to  change.  This  resistance  to  change  is  still 
problematic.  Testing  philosophies  developed 
for  monolithic  developments  must  become 
more  flexible  and  dynamic,  with  redefinition 
of  test  phases,  operational  and  baseline 
capabilities  and  thresholds  to  support  EA. 
The  Logistics  Review  process  must  be 
modified  to  accept  EA,  and  pennit 
alternative  concepts  of  logistics  support.  The 
DOD  acquisition  process  must  more 
completely  endorse  EA  as  an  efficient  way 
of  doing  business,  and  provide  requisite 
documentation  to  support  and  allow  a 
flexible  approach  to  program  development 


where  that  approach  is  appropriate.  These 
changes  would  all  improve  this  already 
efficient  method  of  Defense  Acquisition. 

CONCLUSIONS/SUMMARY 

Acquisition  of  OSS  using  an 
Evolutionary  Acquisition  Strategy  is  being 
successfully  implemented.  Despite  rigid 
mindsets  by  well  meaning  officials  in  the 
acquisition  and  logistics  communities  and 
some  unnecessary,  inefficient  resource 
allocation  to  standard  acquisition  program 
requirements,  most  obstacles  have  been 
overcome.  Most  encouraging  have  been  the 
successes  OSS  has  achieved  in  providing 
rapid  deployment  of  state  of  the  art 
technology  to  meet  fleet  requirements.  This 
has  been  achieved  chiefly  through  (1)  the 
open  minded  participation  of  the  user  and 
test  communities,  (2)  the  innovative,  patient, 
persistent,  and  bold  leadership  of  the  OSS 
Project  Manager,  and  (3)  the  support  of  the 
Program  Manager,  Program  Director, 
Program/Resource  Sponsor,  in  particular, 
VADM  Jerry  Tuttle  (N6),  and  the  N62  team, 
and  ASN  (RDA). 

Evolutionary  Acquistion  is  the  most 
effective  approach  to  acquiring  C4I  systems 
and  is  being  successfully  impleemnted  by 
SPAWARSY'SCOM  to  acquire  systems  such 
as  the  Operations  Support  System. 
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ABSTRACT 


Computer  technology  has  reached  a  level  of 
maturity  where  virtual  prototyping  can  be  a 
major  benefit  to  the  DOD  acquisition 
process.  Virtual  prototyping  is  a  computer 
based  simulation  with  a  degree  of  functional 
realism  comparable  to  a  physical  prototype. 
Used  for  concept  definition,  idea  exchange 
with  users,  product  design,  test  and 
evaluation  and  as  a  decision  tool,  virtual 
prototyping  is  applicable  to  both  major  and 
less  than  major  systems  and  their 
components. 

Virtual  prototyping  is  an  automated  method 
that  can  maximize  the  benefits  of  integrated 
product  teams  through  the  use  of  scientific 
data  visualization,  3-dimensional  drawings 
and  animated  simulations.  It  facilitates 
rapid  multi-discipline  communication,  which 
translates  to  more  robust  designs,  lower 
design  and  manufacturing  costs  and 
compressed  cycle  time.  Virtual  prototyping 
facilitates  the  DOD  Dual  Use  thrust  because 
of  its  electrical  transportability,  the  reuse  of 
designs  and  the  ability  to  rapidly  reconfigure 
system  characteristics  throughout  the 
engineering  design  process. 


The  fidelity  of  existing  3-dimentional  virtual 
prototyping  systems  has  reached  the  level  of 
confidence  where  major  defense  contractors 
are  foregoing  physical  mockups.  Virtual 
prototyping  is  seen  as  one  of  the  best  ways 
to  reduce  costs  and  gaining  an  essential 
competitive  advantage  in  the  21st  Century. 

Our  review  of  existing  DOD  acquisition 
policies  indicates  there  is  flexibility  to 
accommodate  virtual  prototyping  principles, 
but  to  date  it  appears  that  few  program 
managers  have  t^en  advantage  of  this 
value-added  process.  How  to  maximize  the 
use  of  virtu^  prototyping  in  DOD  warrants 
further  investigation.  Virtual  prototyping 
has  the  potential  for  providing  DOD 
decision  makers  the  ability  to  select 
optimum  weapon  systems  based  on  their 
impact  in  an  electronic  battlefield  simulation 
-  the  "marketplace"  of  the  future. 
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ABSTRACT 


The  Department  of  Defense 
acquisition  process  needs  re-evaluation  and 
re-adjustment.  The  need  for  this 
adjustment  centers  around  tremendous 
change  and  competition  in  the  world 
marketplace.  These  dynamic  forces  require 
that  we  seriously  rethink  how  we  acquire 
new  weapon  systems.  In  this  paper  we 
analyze  changes  in  the  defense  acquisition 
environment  and  make  suggestions  for 


improvement.  The  changes  we  focus  on 
include  the  acquisition  process  itself, 
optimizing  privatization  and  competition 
efforts,  innovative  entrepreneurial 
approaches  to  defense  acquisitions,  and 
Total  Quality  Management  in  the  defense 
acquisition  arena.  Finally,  we  preview 
national  security  and  defense  positions 
sponsored  by  the  new  Democratic 
Administration. 


THE  NEED  TO  REEVALUATE  AND  REALIGN  DEFENSE  ACQUISITIONS 


With  the  transformational  changes 
occurring  internationally  and  in  the  United 
States,  it  becomes  imperative  that  we 
research,  innovatively  redesign,  and 
implement  new  defense  acquisition 
strategies  in  order  to  be  able  to  optimize 
both  private  and  public  sector  contributions 
to  our  national  defense  system.  Abroad  we 
find  significant  contributions,  like  those  of 
Japan  to  our  high  technology  weajwn 
systems. (1)  The  Soviet  Union  has 
crumbled,  leaving  questions,  even  after 
START  n,  regarding  the  control  and 
disposition  of  significant  weapon  systems 
left  over  from  the  former  USSR.  The 
Warsaw  Pact  has  vanished,  ostensibly 
requiring  less  commitment  on  the  part  of 
the  United  States  for  support  of  its  NATO 
allies.  The  Gulf  War,  manifested  primarily 
by  Operation  Desert  Shield  and  Desert 
Storm,  proved  beyond  a  doubt  that  current 
U.S.  weapon  systems  are  both  advanced 
and  lethal  in  terms  of  efficiency  and 
effectiveness  when  compared  to  other 
systems  in  the  world.  This  conflict  also 
demonstrated  that  U.S.  military  forces  are 
exceptionally  skilled  in  their  profession.  In 
1992,  the  European  Common  Market  began 
to  cause  concern  about  increasing 
competition  from  Europe,  while  the  North 
American  Free  Trade  Agreement  forced  us 


into  different  modes  of  competition  in 
North  America. 

Competition  has  become  one  of  the 
primary  themes  of  the  decade.  Increased 
efficiency  and  effectiveness  have  permitted 
both  private  and  public  sector  managers  to 
do  more  with  less.  Change  is  occurring  at 
all  levels  of  both  the  private  and  public 
sectors. (2  &  3)  Traditional  blue  chip 
corporations  such  as  General  Motors  and 
IBM  have  cut  back  substantially  on 
facilities  and  payroll. (4  &  5)  The  U.S. 
military  is  reorganizing  and  downsizing 
approximately  30%  in  just  a  few  years,  and 
the  defense  industry  as  we  have  known  it  is 
being  transformed.  (6)  Employment  is 
projected  to  plummet  by  300,000  in 
1993.(7)  There  have  been  numerous 
studies  of,  recommendations  for,  and 
adjustments  towards  more  competition 
within  the  DoD  and  the  military  acquisition 
system. (8)  A  significant  portion  of  the 
contraction  in  defense  will  come  from 
procurement  by  way  of  defense  contract 
reductions.  The  recent  radical  downsizing 
is  exacerbated  by  a  cyclical  downturn  and 
by  continued  procurement  reforms. (9)  Yet, 
while  the  Pentagon,  the  White  House,  and 
Congress  all  relied  on  competition  to  solve 
the  fraud,  waste  and  efficiency  problems  in 
the  procurement  system,  we  now  find  that 
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perhaps  the  major  change  was  a  drop  in  evaluate  the  world  of  defense  and  military 

contractor  profits,  while  fraud  and  waste  acquisitions,  and  re-align  and  re-design  it 

can  still  be  found.  (10)  so  it  can  be  synchronized  with  the  new 

world  we  will  find  as  we  approach  the  mid- 
Such  are  the  forces  that  come  into  1990s.  In  particular,  in  this  manuscript  we 

play  when  analyzing  how  to  adjust  the  will  explore  developments  unfolding  in 

military  acquisition  system  towards  more  defense  acquisition,  privatization,  and 

optimal  performance  in  a  turbulently  competition,  innovative  entrepreneurship 

dynamic  and  changing  world.  Both  and  Total  (Quality  Management.  We  will 

internationally  and  domestically,  we  are  then  extract  lessons  for  adjusting  the 

witnessing  nothing  short  of  a  "paradigm  defense  acquisition  system  so  that  it  can 

shift. "(11)  In  order  for  us  to  be  able  to  work  more  optimally  in  the  new  world  we 

support  the  transformed  defense  mission  in  find  emerging  in  mid-  1990s. 

this  country,  we  will  have  to  completely  re- 

CHANGES  IN  DEFENSE  ACQUISITIONS 


Current  changes  in  the  defense 
acquisition  world  are  rapid  and  dynamic. 
These  changes  are  occurring  on  many 
fronts  ranging  from  privatization  and 
entrepreneurship  to  TQM.  The  impact  of 
these  changes  is  significantly  affecting  the 
defense  acquisition  process. 

Maintain  The  Industrial  Base 

Future  weapon  systems  acquisitions 
will  certainly  change.  Downsizing  caused 
by  diminished  enemy  threats,  fiscal 
resource  constraints,  increased  operational 
life  of  major  weapon  systems,  and 
escalating  costs  of  new  systems  are  all 
major  factors  influencing  the  defense 
acquisition  system  of  the  future.  (12)  The 
"pipeline"  procurement  process  is  changing 
and  being  replaced  with  several  new  and 
innovative  approaches  to  defense 
acquisitions.  (13)  The  underlying  theme  in 
these  new  approaches  is  an  attempt  to 
maintain  the  defense  industrial  base.  (14) 
The  DoD  has  four  main  objectives  for 
maintaining  the  industrial  base: 

1 .  It  must  support  the  basic 
peacetime  force  structure  level. 

2.  During  wartime,  it  must  be  able 
to  provide  weapons  during  times  of  crisis. 

3.  It  must  provide  rapid  production 
capacity  to  meet  a  global  threat. 


4.  It  must  be  efficient  and  cost 
effective.  (15) 

The  new  Secretary  of  Defense,  Les 
Aspin,  former  Democratic  Chairman  of  the 
House  Armed  Services  Committee,  has 
proposed  some  measures  to  maintain  this 
industrial  base  despite  turbulent  changes 
affecting  this  industry.  His  plan  includes 
selectively  upgrading  weapon  systems  and 
selectively  procuring  new  weapon  systems 
at  a  low  production  rate  in  order  to  keep 
critical  defense  suppliers  in  business. 
Additionally,  the  new  Secretary  intends  to 
prioritize  "silver  bullet"  procurements  high 
on  the  acquisition  procurement  priority  list. 
These  "silver  bullet"  procurements  will 
include  technologically  superior  weapons 
systems  that  will  act  as  force  multipliers 
during  conflicts.  (16) 

Another  major  aspect  of  Aspin 's 
plan  to  maintain  the  industrial  base  revolves 
around  research  and  development.  This 
aspect  is  called  "Rollover  Plus."  It 
encompasses  multiple  research  and 
development  cycles  leap  frogging 
technological  progress  over  technological 

generations  without  having  multiple 

production  programs.  The  "plus"  refers  to 
upgrading  manufacturing  technology, 

operations  testing,  and  concurrent 

engineering  programs. (17)  The  drawback 
with  the  "rollover  plus"  concept  is  that  no 
strong  financial  incentives  currently  exist  to 
encourage  defense  manufacturers  to 
concentrate  on  research  and  development 
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while  omitting  or  significantly  cutting  back 
on  the  production  cycles.  Typically,  the 
production  line  provides  def^ense 
manufacturers  with  their  profit 
incentive.  (18) 

Flexible  Acquisition  Strategy 

The  new  defense  acquisition 
strategy  includes  many  new  ways  to  make 
DoD  procurements.  However,  none  of 
them  embrace  the  long-standing  tradition  of 
the  "pipeline"  procurement  process. 
Instead,  new  procurements  embody 
flexibility  and  adaptability.  This  new 
flexible  acquisition  strategy  will  still  use 
military  oriented  research  and  development, 
but  the  DoD  will  also  use  commercial 
technologies  when  possible.  Additionally, 
some  acquisition  proponents  endorse  using 
"shelved  technology"  where  a  weapon 
system  is  develops  through  the  prototype 
phase,  including  working  out  the  defects, 
and  then  is  stoi^  until  it  is  needed.  Only 
if  the  product  is  needed  does  it  ever  enter 
the  production  pipeline.  Multiple  research 
and  development  cycles  will  also  be  used  to 
keep  us  technologically  ahead  of  any  enemy 
without  incurring  the  production  costs. 
Instead,  our  defense  industry  will  be 
working  on  new  generations  of 
technologies  from  prototypes  of  older 
generation  technology  that  are  still 
significantly  more  advanced  than  those  of 
our  enemies. 

The  true  goal  of  this  new,  flexible 
acquisition  strategy  is  to  assemble  and 
deploy  new  weapon  systems  during 
mobilization  and  build-up  of  our  forces 
during  a  conflict.  This  just-in-time 
inventory  methodology  can  be  an  extremely 
effective  and  efficient  method  for  weapon 
acquisitions.  (19)  However,  it  is  dependent 
on  the  ability  of  defense  manufacturers  to 
get  new  weapon  systems  through  the 
production  line  in  a  timely  way.  Many 
analysts,  however,  believe  that  the 
manufacturing  technology  of  our  defense 
industry  is  not  up  to  this  challenge.  For 
example,  a  Rand  Corporation  study 
identified  several  weaknesses  in  the  defense 
manufacturing  industry.  These  analysts 
cited  the  need  for  improved  production  and 
plant  operations  to  include  computer  aided 


manufacturing.  This  upgraded 

manufacturing  technology  could  increase 
production  flexibility  and  help  the  defense 
firms  reach  their  profitability  goals  even 
with  reduced  procurements  by  having  high 
production  efficiencies  despite  low 
production  quantities. (20) 

The  DoD's  new  defense  acquisition 
strategy  embodies  most  of  the  above 
mentioned  ideas.  Their  new  strategy  is 
designed  to  ensure  critical  manufacturing 
capabilities  will  be  maintained  despite  a 
reduction  in  procurements.  Auequate 
production  capacity  would  remain  a 
priority.  The  DoD  still  plans  to  maintain 
production  lines  with  significant  spending 
levels  through  the  decade.  Spending  on 
new  weapon  systems,  however,  will  be 
made  very  judiciously  using  the  following 
criteria; 

1 .  The  technologies  have  been 
demonstrated,  tested,  and  proven  to  be 
manufacturable. 

2.  There  clearly  is  a  military  need 
for  the  new  system. 

3.  The  new  system  is  cost 
effective. 

Systems  that  meet  these  criteria  will  go 
beyond  the  research  and  development  level 
and  enter  into  the  production  phase  of 
acquisitions.  (21)  Although  the  quantity  of 
new  weapons  produced  will  decrease,  new 
key  technologies  referred  to  as  "silver 
bullets"  by  the  defense  establishment,  will 
be  redesigned  to  maintain  our  technological 
advantages.  They  will  drive  DoD  efforts 
to  develop  and  produce  new  and  innovative 
technology. 

The  new  DoD  acquisition  strategy  is 
designed  to  promote  flexibility  and 

selectively  procure  new  weapon  systems  to 
maintain  our  technological  edge  over  our 
enemies,  while  still  maintaining  our 

industrial  base.  One  method  to  help 

downscale  the  size  of  the  military 

acquisition  process  is  to  privatize  as  many 
functions  in  the  acquisition  process  as 
possible. 
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PRIVATIZATION  AND  COMPETITION 


As  the  combination  of  increasing 
competition,  cutbacks,  and  reorganization 
have  begun  to  take  effect,  alternatives  have 
been  sought  for  the  traditional  acquisition 
service  delivery  and  support  systems.  One 
of  the  most  promising  alternatives  has  been 
fine  tuning  privatization.  By  creating 
competition  between  maintenance  depots 
and  private  firms,  DoD  is  optimizing 
efficiency  during  this  period  of  downsizing 
and  shrinking  budgetary  support.  The  goal 
has  evolved  toward  providing  the  best 
service  for  the  best  price.  The  most 
significant  change  has  been  to  privatize 
some  of  the  maintenance  functions  of  newly 
acquired  defense  hardware.  It  is  also 
important  to  set  up  and  monitor 
competition  between  private  contractors 
and  government  employees  so  that  it  is 
possible  to  determine  equitable  cost 
comparisons.  At  the  same  time  DoD  must 
guard  its  ability  to  provide  the  appropriate 
surge  capability.  The  effect  of  the 
adjustment  has  been  to  improve  overall 
performance  by  monitoring  and  adjusting 
support  costs  and  by  consolidating 
operations.  Additional  privatization 
options  include  system  repair  work  and 
related  DoD  contract  work.  Many 
peripheral  duties  involved  in  acquisitions 
can  also  be  privatized.  These  peripheral 
duties  could  include  security  personnel, 
legal  staffing,  contract  administration 
support,  and  supervision  of  contracts. 
Beyond  these  adjustments,  the  DoD  is 
moving  towards  private  sector  products  and 
processes  where  possible.  This  shift  from 
militarily  unique  products  and  services  will 
both  increase  military  access  to  the  national 
industrial  base  and  support  the  growing 
trend  towards  privatization  of  public  sector 
services,  both  in  this  country(22)  and 
abroad(23)  as  the  political,  military,  and 
industrial  world  changes. 

Purchasing  weapon  systems  and 
equipment  represents  the  largest  category  of 
public  sector  purchases  from  the  private 
sector.  The  contracts  with  private  firms  for 
hardware,  research,  and  logistical  support 
are  an  integral  part  of  our  modem  defense 
structure.  One  third  of  the  1987  defense 


expenditures  of  $295  billion  was  spent  on 
DoD  personnel.  The  remaining  $186 
billion  went  for  goods  and  services 
purchased  from  the  private  sector.  Most 
was  spent  on  procurement  of  goods  like 
hand  tools,  office  supplies,  cars,  and  many 
other  commercially  produced  goods. 
However,  private  sector  services  that  are 
significant  included  transportation,  research 
and  development,  base  management,  and 
administrative  support.  The  money  spent 
here  was  more  than  the  total  payroll  for  all 
of  our  uniformed  personnel  in  the  armed 
services  ar  ‘  over  twice  the  payroll  for  civil 
service  defense  employees.  (24) 

The  system,  because  of  its  size  and 
impact,  draws  a  lot  of  attention  and 
criticism.  Typical  complaints  over  time 
include  the  following: 

1.  The  system  delivers  too 
little  for  the  amount  of  money  paid  and 
accountability  is  not  adequately  enforced. 

2.  The  system  often  builds 
weapons  for  the  wrong  wars,  has 
continuous  development  and  production 
delays,  and  has  significant  and  recurring 
quality  problems. 

3.  Accountability  is 
seriously  lacking.  Criticism  is  most 
notably  focused  on  costs,  schedules  and 
standards.  (25) 

There  is  also  a  perception,  among 
the  public,  that  private  contractors,  the 
military,  and  politicians  working  together 
often  ignore  the  public  interest  and  make 
mutually  beneficial  agreements  that  do  not 
optimize  military  ne^s.  Such  coalitions 
are  referred  to  in  academia  as  "iron 
triangles."  While  some  would  argue  that  it 
is  virtually  impossible  to  reform  the 
system,  there  have  been  numerous 
congressional  findings  and  military  reports 
(26)  over  the  last  decade  calling  for 
increased  competition  to  solve  the 
problem.  (27)  There  have  also  been 
hundreds  of  legislative  proposals  for 
procurement  reform.  And  the  mid-1980s 
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witnessed  landmark  legislation  such  as:  the  and  development  functions  right  through  to 

Competition  in  Contracting  Act,  Warranty  production  and  beyond  to  maintenance,  is 

Requirements  in  Appropriations  Bills,  in  order.  There  is  a  history  of  significant 

contractor  cost  limitations,  and  reduced  public  sector  involvement  in  this  arena.  (32) 

allowable  profit  margins.  Perhaps  the  balance  between  private  sector 

and  public  sector  functions  can  or  should  be 

Many  would  argue  that  the  adjusted,  given  the  transformed  national 

subsequent  increased  competition  has  security  environment  we  find  in  1993. 

solved  a  major  part  of  the  problem.  Others  Should  we  end  up  with  fewer  viable 

point  to  the  increased  costs  and  other  defense  companies  to  provide  meaningful 

tradeoffs,  like  time,  involved  in  competition  after  the  shake-out,  a  re¬ 
establishing  more  competitive  systems.  evaluation  and  re-alignment  of  the  private- 

The  reality  of  the  very  limited  number  of  public  sector  balance  would  be  in  order, 
prime  defense  contractors  and  the  enormous 

amount  of  money  to  be  invested  at  Even  the  pro-military  Bush 

competitive  bids  is  a  factor  to  be  dealt  Administration  acknowledged  that  private 

with. (28)  Since,  in  1993,  we  can  see  a  contractors  waste  billions  of  dollars  due  to 

reduction  of  300,000  private  defense  mismanagement.  There  were  also  calls  for 

industry  employees,  the  prospects  for  increasing  resources  for  hiring  more  federal 

increasing  and  maintaining  meaningful  auditors.  (33)  Given  such  a  state  of  affairs 

competition  becomes  even  more  it  should  be  apparent  that  management  on 

challenging. (29)  Approximately  60%  of  both  sides  of  the  issue  should  take  a  long 

defense  contracts  were  awarded  hard  look,  especially  with  all  the  changes 

competitively  in  1987.  In  1983  it  was  being  experienced  in  the  transformation  of 

about  33%.  The  defense  industry,  in  a  defense  in  America  and  the  rest  of  the 

study  of  its  own  warned  that  if  contract  world, 

guidelines  were  tightened  too  much,  the 

effect  would  be  to  force  more  companies  One  recent  policy  development 

out  of  competition  thereby  limiting  worthy  of  note  is  progressing  in  the 

competition  in  the  future.  (30)  Lundquist  direction  of  increased  accountability 

argues  that  profitability  was  destroyed  by  regarding  private  sector  contractors.  The 

the  reforms  of  the  1980s  and  by  coupling  Single  Audit  Act,  implemented  in  1984, 

the  building  and  research  and  development  (34)  was  designed  to  improve  the 

phases.  Fixed  price  contracts  cost  management  of  federal  fiscal  resources  in 

taxpayers  too  much  and  the  industry,  also  state  and  local  governments.  Concurrently, 

reacting  to  the  downsizing  forces,  will  private  CPA  firms  were  contracted  to  assist 

experience  a  serious  shake-out  with  the  in  this  function.  The  1984  Act  was  applied 

survivors  being  those  defense  companies  to  monitor  the  auditing  function.  Recently, 

which  are  the  smartest  in  their  downsizing  it  has  become  apparent  that  the  1984  Act 

strategies. (31)  has  been  extended  to  audits  of  businesses 

contracting  with  the  government  as 

One  area  ripe  for  rethinking  is  how  well. (35)  This  is  an  evolutionary  step 
private  or  how  public  the  acquisition  which  could  provide  a  significant 

system  either  needs  or  ought  to  be.  It  may  mechanism  to  monitor  privatization  efforts 
be  that  upon  analysis  under  the  currently  in  the  defense  acquisition  system  in  the 

changing  world  situation,  that  a  future, 

realignment  of  the  system  from  its  research 

EVNOVATIVE  ENTREPRENELTUAL  CONTRIBUTIONS 

With  the  promise  of  transformation  acquisition  system  if  we  are  to  optimize 

of  the  national  security  arena  and  our  innovation,  creativity  and  responsibility.  It 

defense  establishment,  we  will  need  to  has  been  about  fifteen  years  since  the  voters 

make  significant  adjustments  in  our  launched  the  taxpayer  revolt  with 
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California's  Proposition  13  on  June  6, 
1978.  Having  local  property  taxes  cut  in 
half,  then  watching  the  conservative 
approach  to  government  from  the  Reagan 
Administration  and  the  recession  of  1982, 
government  managers  and  administrators 
had  no  choice  but  to  redesign  their 
approach  to  responsibly  deliver  their 
services.  (36)  In  November  1992,  Colorado 
passed  a  similar  measure,  known  as 
Amendment  1 ,  (37)  indicating  that  the 
revolt  is  still  underway.  Americans  want, 
and  are  demanding  more  quality  services 
for  less  money,  and  private  sector  concepts 
of  quality,  efficiency  and  entrepreneurship 
are  becoming  increasingly  important.  (38) 

Fewer  services  or  higher  taxes  are 
among  the  obvious  answers.  But 
Americans  are  demanding  more.  And,  as 
public  managers  have  begun  to  recover 
from  the  fiscal  and  service  demand 
assaults,  their  innovative  adjustments  have 
begun  to  take  root  demonstrating 
improvements  in  both  quality  and  quantity. 
The  emerging  paradigm,  however,  is  very 
different  than  the  lethargic  and 
unresponsive  bureaucracies  have  offered  in 
the  past.  The  new  paradigm  could  be 
described  as  decentralize,  flexible, 
adaptive,  competitive,  learning,  customer 
oriented,  creative,  lean,  and 
streamlined.  (39) 

Forced  to  change  the  way  they  went 
about  the  business  of  government,  public 
managers  developed  a  new  model  of 
government.  It  included  both  managerial 
and  governance  concepts  that  would  permit 
them  to  cover  the  legal  mandates  while 
innovatively  and  at  times  aggressively, 
attacking  traditional  bureaucracy  as  they 
launch  into  a  new  era  of  delivering 
government  services.  New  approaches  and 
concepts  such  as  contracting  out, 
privatization,  (40)  public-private 
partnerships,  load  shedding,  alternative 
service  delivery,  participatory  management, 
empowerment,  and  total  quality 
management  were  tried  and  accepted  as 
they  proved  useful.  As  we  move  well  into 
the  mid-nineties,  there  is  an  increasingly 
large  list  of  successful  experiments  at  all 
levels  of  government.  Some  are  identifying 


the  change  as  a  paradigm  crisis  in  the 
classical  Thomas  Kuhn  paradigm  shift 
sense.(41) 

As  President  Clinton  takes  over  as 
the  leader  of  the  national  government  in 
1993  we  already  have  some  clues  as  to  how 
he  intends  to  approach  delivering 
government  services  and  products  to  more 
people  at  higher  quality  while  attempting  to 
reduce  the  deficit.  He  has  already  endorsed 
a  host  of  cutting  edge  concepts,  instruments 
and  approaches  delineated  in  a  book  by 
David  Osborne  and  Ted  Gaebler  titled  Re- 

Inventing  Government: _ How  the 

Entrepreneurial  Spirit  is  transforming 
Government.  (42)  On  the  cover  of  the  book 
Clinton  is  quotexl:  "This  book  should  be 
read  by  every  elected  official  in  America. 
Those  of  us  who  want  to  revitalize 
government  in  the  1990s  are  going  to  have 
to  reinvent  it.  This  book  gives  us  a 
blueprint."  Senator  Sam  Nunn  also  has  an 
endorsement  on  the  cover.  A  Time  article, 
after  the  election,  entitled  "A  Prophet  of 
Innovation"  describes  author  David 
Osborne  as  one  of  Clinton's  people... to  be 
consulted  on  how  to  reinvent  government 
and  fight  bureaucratic  bloat.  (43) 

Endorsements  such  as  these,  made  by 
policy  makers  who  will  have  an  important 
impact  on  defense  acquisitions,  are  worthy 
of  note. 

Osborne  and  Gaebler  present 
numerous  military  and  non-military 
examples  of  doing  more,  innovatively  and 
creatively,  with  increasingly  fewer 
resources.  In  brief,  they  set  forth  ten 
principles,  reconstructed  as  follows: 

1.  Alternatives  to  in-house  service 

delivery  should  be  explored  to  find  more 
efficient  alternatives.  Vouchers, 

contracting  out,  volunteers,  and  public- 
private  partnerships  are  just  a  few  ideas 
here. 

2.  Competition  should  become  a 
greater  factor  wherever  it  can  be  effectively 
infused. 

3.  Management  teams  and 
governing  councils  can  and  should  be 
empowered  to  take  on  many  program 
responsibilities. 
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4.  Rules  should  be  dramatically 
reduced.  Budgets  should  be  re-designed 
and  program  management  streamlined. 

5 .  Performance  measurement 
should  be  re-thought  and  oriented  towards 
performance  and  policy  outcomes  as 
opposed  to  program  inputs. 

6.  Governments  can  and  should 
earn  money  via  fees,  internally  competitive 
profit  centers,  and  shared  savings,  etc. 

7.  Governments  should  set  up 
market  systems  where  appropriate. 

8.  Clients  are  customers.  Survey 
them.  Find  out  what  they  need  and  want. 
Train  employees  to  serve  customers.  Aim 
towards  service  qualities  and  conveniences. 

9.  Decentralization  through 
teamwork,  participatory  management, 
quality  circles,  employee  development 
programs,  and  labor  management  efforts 
should  be  optimized. 

10.  Prevention  should  become  a 

priority.  Assertively  forestalling 

TQM  IN  THE  NEW 

The  U.S.  military  is  embracing  the 
concepts  of  Total  Quality  Management  with 
open  arms.  With  a  strong  customer  focus, 
continuous  process  improvement,  and 
genuine  concern  for  people  quality 
management  improvements  should  have  a 
firm  foundation  in  the  acquisition  process. 
Quality  management  driven  improvements 
can  help  the  DoD  towards  its  goal  of  an 
efficient  and  cost  effective  industrial  base 
with  high  efficiencies  despite  low 
production  levels. 

In  July,  1992  the  U.S.  Air  Force 
completed  a  major  reorganization  designed 
to  improve  efficiency  and  improve 

customer  satisfaction  within  their 

acquisition  system.  Before  this 

reorganization  the  acquisition  process,  from 
Milestone  0  through  Milestone  3  and  part 
of  Milestone  4,  was  controlled  by  the  Air 
Force  "Acquisition  Specialists"  in  Air 
Force  Systems  Command  (AFSC). 
Typically,  once  the  new  weapon  system 
made  it  past  full  scale  production  and  into 
the  operations  and  support  phases  of  the 
acquisition  process,  transfer  of 

responsibility  for  acquiring  and  maintaining 


maintenance,  health,  and  trash  problems  in 
advance  would  be  appropriate 
examples.  (44) 

Osborne  and  Gaebler  are  clearly  at 
the  cutting  edge  of  entrepreneurial 
management  in  the  context  of  delivering  the 
services  and  goods  of  government 
efficiently  and  effectively.  Many  of  their 
proposals  would  not  be  appropriate  for 
acquisitions.  A  great  number  of  them 
would  be,  however.  There  are  some 
professionals  who  address  the  governance 
caveats  involved  in  such  adjustments  as 
well.  Johnston  has  examined  many  of  the 
trade-offs  in  privatizing  government 
management  and  has  rais^  some  of  the 
sovereignty  and  governance  concerns  as 
well. (45)  And,  in  a  review  of  the  Osborne 
and  Gaebler  book,  Charles  T.  Goodsell 
offers  "Goodsell 's  Ten  Principles  of 
Rediscovering  Government.  "(46) 


DEFENSE  STRATEGY 

that  wea[wn  system  was  made  to  the 
"Maintaining  and  Supply  Specialists"  in  Air 
Force  Logistics  Command  (AFLC).  This 
transfer  of  responsibility  caused  a  lot  of 
frustration  for  manufacturers,  users,  and 
maintainers  of  these  new  weapon  systems 
because  there  was  no  single  point  of  contact 
for  weapon  systems  questions  and 
problems. 

Because  of  the  inherent 
inefficiencies  of  having  two  different 
commands  managing  a  weapon  system, 
depending  on  the  weapon  system  life  cycle 
stage,  the  Air  Force  combined  these  two 
organizations  into  a  newly  formed 
command  called  Air  Force  Materiel 
Command  (AFMC).  AFMC  is  a  large  Air 
Force  command  with  125,000  people  and 
50%  of  the  Air  Force  budget.  (47) 
AFMC's  mission  is: 

"Through  integrated  management  of 
research,  development,  test,  acquisition  and 
support,  AFMC  advances  and  uses 
technology  to  acquire  and  sustain  systems 
in  partnership  with  the  customers. "(48) 
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The  merging  of  AFLC  and  AFSC 
was  completed  on  July  1,  1992.  This  new 
organization  procuring  new  weapons 
systems  for  the  Air  Force  has  integrated 
several  acquisition  process  areas  into  a  new 
management  approach  called  "Integrated 
Weapons  System  Management.  "(49)  This 
new  focus  of  integrating  procurement, 
supply,  and  maintenance  of  weapon 
systems,  all  under  the  control  of  one 
director,  has  helped  increase  responsibility 
among  acquisition  personnel  who  manage 
an  entire  program  over  it's  entire  life, 
rather  than  shifting  responsibility  for 
program  management  to  some  other 
organization  part-way  through  the  weafK)n 
system  life  cycle.  The  Integrated  Weapons 
System  Management  Program  has  three 
components: 

1.  "Cradle  to  grave"  management 
of  weapon  systems  so  that  acquisition 
personnel  consider  the  entire  product  life 
cycle  in  decisions. 

2.  Provide  customers  with  one 
focal  point,  one  source  of  information  and 
decisions  in  the  Air  Force. 

3.  Create  a  seamless  organization 
responsible  for  eight  critical  processes 
ranging  from  managing  funding  for  a  new 
weapon  system  to  upgrades  for  it.  (50) 

Total  Quality  Management  ideas 
have  been  incorporated  into  the  restructure 
of  AFSC  and  AFLC.  This  approach  to 
quality  improvement  was  used  to  form 
AFMC.(51)  The  new  commander  of 
AFMC,  General  Yates,  believes  that  all 
workers  must  have  decision  making 
authority  in  the  acquisition  process  with  the 
power  to  change  obstacles  to  making 
quality  products.  An  extended  delineation 
of  the  exact  methods  and  techniques  used 
by  AFMC  to  implement  this  charter  is 
beyond  the  scope  of  this  manuscript. 

Defense  acquisition  processes  have 
many  quality  management  programs  built 
in.  The  bulk  of  the  processes  lies  in  the 
hands  of  the  contractors  that  actually 
produce  the  weapon  systems.  The  DoD  has 
openly  endorsed  quality  management 


improvements  with  its  contractors. 
However,  the  efficiency  of  these  quality 
management  programs  is  questionable.  For 
example,  at  one  large  defense  contractor, 
one  senior  official  stated  that  his  company 
has  a  ..."TQM  program  to  get  awarded 
government  contracts  and  [that]  the 
program  is  mostly  eyewash  for  government 
officials.  "(52)  Nonetheless,  Air  Force 
Logistics  Command  won  the  President's 
Award  for  Quality  and  Productivity 
Improvement  in  a  government- wide 
competition  in  1991.  Since  then,  the 
Defense  Department  is  well  on  its  way 
towards  adopting  and  internalizing  TQM  as 
the  way  it  does  business.  Integrating 
AFLC  and  AFSC  into  AFMC  is  probably 
the  most  significant  application  of  TQM 
since  former  Secretary  of  Defense,  Frank 
Carlucci  adopted  TQM  as  the  way  the 
Pentagon  would  do  business.  (53) 

The  pivotal  role  of  quality 
improvement  efforts  in  acquisitions  lies  in 
establishing  contracts  with  defense 
manufacturing  firms.  The  DoD  has  been 
writing  contractual  clauses  covering  "value 
engineering.”  Value  engineering  is  a 
concept  embodied  contractual  provision  that 
provides  financial  incentives  to  defense 
manufacturers  to  find  manufacturing 
process  improvements  in  already 
established  contracts.  The  defense  finn 
then  gets  a  share  of  the  savings  to  the 
government  when  the  process  is  modified 
to  incorporate  the  suggested  process 
improvements.  Clearly,  value  engineering 
has  many  potential  quality  improvement 
gains  provided  the  program  is  nm 
competently.  ITiis  program  has  been 
around  for  almost  two  decades  and  has 
often  been  found  to  be  ineffective.  (54)  It 
would  therefore  have  to  be  re-emphasized 
and  more  closely  monitored  in  order  for  it 
to  achieve  its  full  potential. 

The  DoD  has  cited  several  problems 
with  the  value  engineering  program  which 
could  be  improved  upon.  First,  is  a  lack  of 
awareness  among  government  and 
industrial  officials  that  the  program  exists. 
Often,  those  who  manufacture  new  weapon 
systems  are  not  aware  of  the  value 
engineering  contractual  provisions. 
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Second,  government  and  defense  industry 
workloads  are  too  high.  Many  officials  and 
workers  simply  do  not  have  the  time  nor 
the  motivation  to  look  for  optional  cost 
savings.  Third,  the  government  is  slow  to 
evaluate  and  process  those  improvement 
proposals  which  have  already  been 
submitted.  Sometimes  it  has  taken  over 
twice  the  4,S  day  target  time  to  get  the 
submitted  proposal  acted  upon  by 
goveniment  officials.  Past  inefficiencies  in 
managing  the  .system  provide  disincentives 
for  contractors. 

The  DoD,  under  President  Bush, 
implemented  .some  admini.strative  changes 
designed  to  improve  the  acquisition 
process.  He  established  the  Defense 

NATIONAL  SECURITY  AND 

What  do  we  know  about  how 
President  Clinton  and  his  new  Secretary  of 
Defense,  Les  Aspin,  will  approach  national 
security,  and  the  world  of  defense 
acquisition.s?  In  September  1992, 
Georgetown  University  released  its  analysis 
through  its  National  Security  Studies 
Programs  and  Global  Security  Project. 
Under  the  leadership  of  Loren  B. 
Thomp.son,  a  report  entitled:  "Bill 

Clinton’s  Views  on  National  Security;  An 
Assessment"  was  released.  Highlights  of 
this  report  provide  a  relevant  frame  of 
reference  for  analyzing  defense  acquisitions 
over  the  next  .several  years. 

President  Clinton  laid  the 
foundation  for  his  future  defense  policy  in  a 
December  12,  1991  .speech  at  Georgetown 
University.  In  his  speech,  "A  New 
Covenant  for  American  Security"  he  noted 
that  America  has  two  great  foreign  policy 
challenges;  l)defining  a  new  national 
security  policy  based  on  the  Cold  War 
victory;  and  2)  generating  a  new  economic 
policy  related  to  a  new  era  of  global 
growth.  Surveying  existing  and  potential 
capabilities  and  threats,  Clinton's  approach 
is  matched  against  Bush's.  Clinton  favors  a 
slightly  smaller,  more  conventional,  and 
more  U.S.  ba.sed  force.  He  also  seems 
more  preparetl  to  spend  dollars  on 
supporting  the  defense  indu.strial  ba.se. 


Management  Review  to  streamline  the 
acquisition  process  in  1988.  This  high- 
level  panel  has  been  effective  in  increasing 
efficiencies  in  some  areas  of  acquisitions. 
For  example,  acquisition  decisions  that 
u.sed  to  take  several  months  later  were 
made  in  about  48  hours.  And,  redundant 
regulations  have  been  modified  or 
removed.  This  review  panel  has  akso 
mandated  reduced  headquarters  staffing  in 
some  military  organizations  to  increase 
efficiencies  as  well.  Despite  these 
improvements  there  is  still  much  room  for 
better  performance.  For  example,  in  1992 
Congress  required  734  one-time  reports  on 
acquisitions.  The  Defense  Management 
Review  is  setting  its  efforts  toward 
reducing  this  workload. (55) 

;NSE:  CLINTON  and  ASPIN 

With  Bush's  spending  plans  for  $1.42 
trillion  and  Clinton's  at  $1.36  trillion,  there 
is  only  a  4%  difference  averaging 
approximately  $12  billion  per  annum  for  5 
years  through  1997  according  to 
preliminary  projected  implementation 
plans.  (56) 

Clinton’s  defense  posture  turns  out 
to  be  quite  middle  of  the  road.  His 
primary  advisors  on  defense  have  been 
Senate  Armed  Services  Committee 
Chairman  Sam  Nunn  and  then  House 
Anned  Services  Committee  Chainnan  Les 
A.spin.  President  Clinton  has  also  made  it 
clear  that  he  believes  there  are  many 
redundancies  in  the  U.S.  Amied  Forces  and 
that  a  reorganization  of  the  military  could 
reduce  redundancies,  get  better  teamwork, 
and  save  billions  of  dollars.  Clinton 
believes  a  strong  conventional  force  is 
important  for  our  military  future  and  has 
highlighted  several  conventional  weapons 
programs  as  being  important.  The.se 
include:  CVN-76,  the  next  aircraft  carrier 
(Ninety  Class),  the  Air  Force's  F-22 
fighter,  the  C-17  Airlifter  which  is  the  next 
generation  military  transport,  the  V-22 
Osprey  which  is  the  Marine  Corps'  vertical 
take-off  and  landing  aircraft,  and  the 
Navy's  newest  attack  submarine,  the 
Seawolf.  Clinton  also  supports  the  Aniiy's 
M-l  tank  upgrade  program. 
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President  Clinton  has  supported  Les 
Aspin's  position  as  a  proponent  of  a  strong 
defense  industrial  base  which  becomes 
significant  as  becomes  the  Secretary  of 
Defense.  Look  for  projects  that  enhance 
both  defense  and  economics  infrastructure 
development.  Clinton  also  looked  to  Aspin 
for  more  detailed  ideas  on  how  to  address 
defense  acquisition  and  downsizing  issues. 
Clinton's  policies  may  not  necessarily  be 
that  significantly  different  than  Bush's 
would  have  been  under  the  changing 
circumstances.  His  particular  approaches 
to  implementing  his  policies  will  likely  be 
quite  different  than  the  Bush 
Administration's,  however. 

In  his  assessment,  Georgetown's, 
Loren  Thompson  presented  a  number  of 
conclusions  about  Clinton's  national 

TOWARDS  A  PRUDENT 

This  assessment  leads  one  to 
conclude  that  there  will  have  to  be  changes 
made  in  order  to  optimize  the  adjustments 
to  the  transformations  being  experienced  in 
the  defense  acquisition  system.  The 
management,  policy  and  administrative 
forces  creating  the  turbulence  and  change 
affecting  acquisitions  need  to  be  evaluated 
and  adjust^  in  order  to  maximize 
efficiency,  economy  and  performance. 
Based  upon  the  research  done  for  this 
project,  it  appears  that  Clinton  may  draw 


security  policies.  They  include  the 
following:  1)  Clinton  would  not 

substantially  change  current  defense 
spending  trends.  $60  billion  over  five 

years  is  only  about  a  4%  difference  from 
the  Bush  Administration.  2)  The 
deployment  pattern  of  U.S.  military  forces 
would  change.  Clinton  would  base  more 
forces  in  the  U.S.  requiring  the 
development  of  a  more  significant  airlift 
and  sealift  capability.  3)  Military  missions 
and  roles  are  likely  to  be  significantly 
reorganized  to  avoid  duplication  of 

activities  and  assets.  4)  Conventional 
capabilities  will  be  enhanced  and  strategic 
forces  cut  back.  5)  Key  industrial 

capabilities  must  be  protected  and 

promoting  the  integration  of  defense  and 
commercial  production  to  optimize  greater 
efficiency  and  flexibility  must  be 
accomplished  .(57) 

ACQUISITIONS  POLICY 

much  more  significantly  from  the  recent 
developments  in  management  and  policy 
like  privatization,  TQM,  and  reinventing 
government  with  their  evolving 
entrepreneurial  and  participation  enhancing 
mechanisms.  Great  care  must  be  exercised 
to  include  business  strategy  considerations 
as  well,  in  order  that  the  significant 
transitions  in  acquisition  community  be 
managed  in  such  a  way  that  we  promote  an 
optimally  prudent  policy. 


Note 

"THE  IDEAS  AND  OPINIONS  EXPRESSED  HEREIN  ARE  THOSE  OF  THE 
AUTHORS  AND  DO  NOT  NECESSARILY  REFLECT  THE  VIEWS  OF  THE  UNITED 
STATES  AIR  FORCE,  UNITED  STATES  AIR  FORCE  ACADEMY,  OR  ANY  OTHER 
AGENCY  OF  THE  DEPARTMENT  OF  DEFENSE. " 

This  paper  was  funded  in  part  by  the  Defense  Systems  Management  College,  Fort  Belvoir,  Virginia. 

Dr.  Van  R.  Johnston  is  a  Distinguished  Visiting  Professor  of  Management  and  Strategy  at  the  United  States  Air 
Force  Academy;  and  a  professor  of  Management,  Business,  and  Public  Policy  at  the  University  of  Denver.  His 

Ph.D.  is  from  the  University  of  Southern  California. 

Captain  Robert  J.  D'Amico  is  an  Instructor  of  Management  at  the  United  States  Air  Force  Academy,  He  has  a 
MGA  from  the  University  of  Maryland  and  a  BSBA  from  the  University  of  Denver. 


336 


Endnotes 

1.  Deterrence  in  Decay:  The  Future  of  the  U.S.  Defense  Industrial  Base.  The  Center  for 
Strategic  and  International  Studies,  Washington  D.C.,  May  1989. 

2.  Gleckman,  Howard,  et  al.  "Clinton's  Agenda:  Will  Microsurgery  Get  the  Economy 
Moving?"  Business  Week,  a  McGraw  Hill  Publication,  January  25,  1993,  pp.  28  -  32. 

Harbrecht,  Douglas.  "Drafting  a  Dream  Team:  Clinton's  Presidency  Depends  on  His  Picks," 
Business  Week.  A  McGraw  Hill  Publication,  November  23,  1992,  pp.  28ff. 

3.  Verity,  John  W.  "Deconstructing  the  Computer  Industry,"  Business  Week.  A  McGraw 
Hill  Publication,  November  23,  1992,  pp.  90  -  100.  This  is  but  one  of  many  examples 
available. 

4.  Kerwin,  Kathleen,  et  al.  "Crisis  at  GM:  Turmoil  at  the  Top  Reflects  the  Depth  of  its 
Troubles,"  Business  Week,  a  McGraw  Hill  Publication,  November  9,  1992,  pp.  84  -  86. 

5.  Barker,  Robert  and  Verity,  John  W.  "A  Question  of  Value:  Our  Analysis  Finds  Big  Blue 
May  Be  Cheap,"  Business  Week,  a  McGraw  Hill  Publication,  January  18,  1993,  and  Verity, 
John  W. 

6.  Johnston,  Van  R.  and  D'Amico,  Robert  J.,  "Total  Quality  Management  and  the 
Reorganization  of  the  Air  Force,"  a  National  Conference  Paper  for  the  American  Society  for 
Public  Administration's  53d  National  Conference,  Chicago,  April,  1992. 

7.  Shine,  Eric  and  Bomis,  Amy.  "Defense:  Nowhere  to  Run,  Nowhere  to  Hide,"  Business 
Week,  a  McGraw  Hilt  Publication,  January  11,  1993,  pp.  30  -  31 . 

8.  Rich,  Michael,  et  al.  Improving  the  Military  Acquisition  Process:  Lessons  from  Rand 
Research.  Rand/R-3373-AF/RC,  Santa  Monica,  CA,  February  1986. 

Donahue,  John  D.  "Lessons  from  the  Pentagon”  in  The  Privatization  Decisions:  Public  Ends. 
Private  Means.  New  York:  Basic  Books,  Irwin  Publishers  ,  1989,  pp.  126  -  127. 

Cochrane,  Charles  B.  "DoD's  New  Acquisition  Approach:  Myth  or  Reality?"  Program 
Manager.  July  -  August,  1992,  pp.  38  -  46. 

9.  Lundquist,  Jerrold  T.  "Shrinking  Fast  and  Smart  in  the  Defense  Industry,"  Harvard 
Business  Review.  November-December  1992,  p.  76. 

10.  Ibid.,  p.  78. 

11.  Kuhn,  Thomas  S.  The  Structure  of  Scientific  Revolutions.  2nd  ed.,  Chicago:  University 
of  Chicago  Press,  1970. 

12.  Rich,  Op  Cit. 

13.  Cochrane,  Op  Cit. 

14.  Deterrence  in  Decay:  Op  Cit. 

15.  Air  Force  Restructure.  White  Paper,  September  1991. 

16.  Cochrane,  Op  Cit. 


337 


17.  Ibid. 

18.  Rich,  Op  Cit. 

19.  Cochrane,  Op  Cit. 

20.  Ibid. 

21.  Air  Force  Restructure.  Op  Cit. 

22.  Yockey,  Don.  "Defense  Acquisition"  This  is  a  memo  with  twenty  four  pages  of 
attachments  from  the  Undersecretary  of  Defense  to  top  level  officials  outlining  DoD  changes 
regarding  defense  acquisitions  resulting  from  the  dramatic  changes  in  the  national  security 
environment.  U.S.  Department  of  Defense,  May  20,  1992. 

23.  Savas,  E.S.  "Privatization  in  Post-Socialist  Countries,"  Public  Administration  Review. 
Vol,  52,  No.  6,  November/December  1992,  pp.  573  -  581. 

24.  Donahue,  Op  Cit..  pp.  lOlff. 

25.  Ibid.  pp.  102  -  103. 

26.  Ibid.,  p.  111. 

27.  Ibid.,  p.  112. 

28.  Ibid.,  pp.  112  -  113. 

29.  Shine,  Eric  and  Borris,  Amy.  "Defense:  Nowhere  to  Run,  Nowhere  to  Hide,"  in  the 
manufacturing  section  of  Business  Week.  aMcGraw  Hill  Publication,  January  11,  1993, 

p.  64. 

30.  Donahue,  Op. Cit..  pp.  113  -  114. 

31.  Lundquist,  Op.  Cit..  pp.  78ff. 

32.  Donahue,  Op.  Cit..  p.  124fC. 

33.  Schneider,  Keith.  "Firms  Waste  Billions  Say  U.S.  Auditors,"  The  Denver  Post. 
December  12,  1992. 

34.  31  U.S.C.  7501-7507. 

35.  Granoff,  Michael  H.  "Privatization:  The  Road  to  Federal  Regulation  of  Auditing," 
Accounting  Horizon.  Vol.  6,  No.  4,  December  1992,  pp.  76  -  85. 

36.  Osborne,  David,  and  Gaebler,  Ted.  Reinventing  Government:  How  the  Entrepreneurial 
Spirit  is  Transforming  Government.  New  York:  Addison  -  Wesley  Publishing  Company,  Inc. 
1992,  p.  16. 

37.  Editors,  "Reinventing  government  (sic),"  Perspective  The  Denver  Post.  January  17.  1993, 
Section  D,  P.  1 . 


338 


38.  Ibid. 


Port,  Otis,  et  al.  "Quality,"  Business  Week.  A  McGraw  Hill  Publication, 

November  30,  1992,  pp.  66ff. 

39.  Osborne  and  Gaebler,  Op.  Cit. 

40.  Johnston,  Van  R.  "Privatization  of  Prisons:  Management,  Productivity,  and  Governance 
Concerns,"  Public  Productivity  and  Management  Review.  Vol.  XIV,  No.  2,  Winter  1990, 
Jossey  -  Bass  Inc.,  Publishers,  pp.  189  -  201. 

41.  Kuhn,  Op.  Cit. 

42.  Osborne  and  Gaebler,  Op.  Cit. 

43.  Goodgame,  Dan.  "A  Prophet  of  Innovation:  Author  David  Osborne  Tells  How  to 
Remvent  Government  and  Fight  Bureaucratic  Bloat,"  Time.  Vol.  140,  No.  24, 

December  14,  1992,  pp.  50-51. 

44.  Osborne  and  Gaebler,  Op.  Cit. 

45.  Johnston,  Op.  Cit. 

46.  Goodsell,  Charles  T.  "Reinvent  Government  or  Rediscover  It."  Public  Administration 
Review.  January  -  February  1993,  Volume  53,  No.  1,  pp.  85  -  87. 

47.  Hudson,  Neff.  "Materiel  Command:  Risk  Takers,  Stockpilers  Joining  Forces,"  Air 
Force  Times.  June  29,  1992,  pp.  12  -  13. 

48.  Ibid. 


49.  Hudson,  Neff.  "AFMC  Focus  is  'Cradle  to  Grave'  Management,"  Air  Force  Times.  June 
29,  1992,  p.  16. 

50.  Ibid. 


51.  Ibid. 


52.  An  interview  with  a  senior  engineer,  McDonnell  Douglas  Aircraft  Corporation, 

March  12,  1992.  (Interviewee  wishes  to  remain  anonymous). 

53.  Bond,  David  F.  "Military  Budget  Drawdown  Spurs  Movement  Towards  Increased 
Efficiency,"  Aviation  Week  and  Space  Technology.  December  9,  1991,  pp.  58  -  59. 

54.  Fact  Sheet  Program  Manager's  Notebook.  Published  by  the  Defense  Systems  Management 
College,  December,  1988. 

55.  Ibid. 


56.  Thompson,  Loren  B.  "Bill  Clinton’s  Views  on  National  Security:  An  Assessment,"  The 
Global  Security  Project  of  the  National  Security  Studies  Program.  Georgetown  University, 
Washington  D.C.,  September  28,  1992,  pp.  1  -  24. 

57.  aid 


339 


340 


GBI  Success:  A  Steady  Course  Through  a  Sea  of  Stars 

Using  Proactive  Management 

James  C.  Katechis  and  Reba  S.  Seals 

Ground  Based  Interceptor  Project  Office^  Strategic  Defense  Initiative  Organization 


ABSTRACT 

This  article  addresses  the  successful 
management  concept  "model"  employed  by 
a  major  Strategic  Defense  Initiative  (SDI) 
Program.  The  Army's  Ground  Based 
Interceptor  (GBI)  Project  achieved  all 
technical  objectives,  was  within  7.3  percent 
of  the  original  contract  cost,  and  was 
successful  in  proving  the  stressing 
performance  requirements  for  a  low  cost 
interceptor.  The  concept  model  of 
Integrated  Proactive  Management  (IPM)  is 
the  keystone  of  the  GBI  Program 
Management  philosophy.  The  concept  is 
central  to  the  effective  and  timely  blending 
of  technical,  schedule,  and  cost  parameters 
with  the  effective  use  of  a  precious  limited 
resource,  people.  IPM  suggests  adaptive 
management  tools  and  techniques  used  in  a 
proactive  rather  than  a  reactive  manner. 
Although  there  are  many  different  tools  and 
techniques  used  by  the  GBI  Project  Office 
for  both  internal  and  external  management, 
the  contract  itself  is  a  most  important 
management  vehicle.  The  contract  is  the 
formal  means  of  communication  between 
the  government  and  the  prime  contractor  and 
supporting  industries.  The  appropriate 
amount  of  care  is  taken  in  preparing  the 
prime  contract  and  its  associated  clauses  to 
assure  that  the  requirements  to  manage  the 
program  are  covered. 

This  article  addresses  the  application  of  cost 
management  incentives,  subcontract 
reviews,  make  or  buy  decision  reviews,  and 
the  contractor's  total  life  cycle  cost  estimate 
by  the  GBI  Project  Office  to  help  manage 
and  influence  both  the  prime  and 
subcontractor  elements.  The  article  further 
outlines  a  format  used  by  the  government 
and  contractors  when  recovery  plans  were 
required.  Two  new  activities  involving 
future  applications  in  Make  or  Buy  decisions 
and  a  Risk  Management  Process  are 
discussed. 


INTRODUCTION 

The  project  management  concept  places  full 
line  authority  for  centralized  management 
and  direction  of  a  high  priority  system  in  the 
hands  of  a  single  individual,  the  project 
manager.  The  actual  concept  generally 
involves  a  team  approach  consisting  of  a 
government  component  with  the  project 
office  and  other  staff  and  government 
agencies,  a  prime  contractor  component  with 
associated  subcontractors  and  a  government 
support  contractor  component  emphasizing 
specific  technical  enhancements.  The 
management  approach  promulgated  to  the 
team  is  a  direct  reflection  of  the 
management  style  of  the  project  manager 
and  the  environment  in  which  the  project 
must  function.  The  team  prudently  expects 
frequent  environmental  change  and 
anticipates  project  manager  tenure  with 
subsequent  management  style  consistency. 

The  Ground  Based  Interceptor  (GBI)  Project 
Office,  which  is  a  continuation  of  the 
Exoatmospheric  Reentry  Vehicle  Interceptor 
Subsystem  (ERIS)  Project  Office,  is  an 
example  of  the  process.  The  ERIS  project 
manager,  a  Department  of  the  Army 
Civilian,  was  assigned  in  December,  1984 
and  still  serves  in  that  capacity  for  the  GBI 
Project  Office.  The  PM's  style  of 
management  evolved  immediately  with  the 
drafting  of  a  transition  team,  the  preparation 
of  a  request  for  propos”’  and  the  formulation 
of  a  source  selection  structure  and  process. 
A  PROACTIVE  style  of  management 
became  the  mood  at  the  project  office. 
Independent  thinking  was  expected,  "yes- 
men"  were  discouraged  and  the  Economic 
Man  Theory  of  maximum  benefit  for  the 
dollar  was  required.  Risk  reduction,  along 
with  cost,  schedule  and  technical  realism 
were  embedded  in  the  thought  process.  It 
became  evident  that  from  top  management 
down  it  would  be  a  team  approach  requiring 
active  involvement  from  each  participant. 
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For  all  practical  purposes,  the  program 
began  in  1986  when  the  Project  Office  was 
established,  support  contractors  were 
brought  on  board  and  the  prime  contract  was 
signed  in  January.  The  PM  officially 
published  his  Management  Plan  in  which  he 
stated: 


"We  only  get  one  chance  to  start  the 
ERIS  program  right.  It  is  therefore 
necessary  that  we  now  make  clear 
our  overall  plan  for  managing  ERIS  - 
a  plan  which  defines  the  objectives, 
tools  and  techniques  required  for  a 
"proactive"  team.  The  SDC 
commander  has  placed  special 
emphasis  on  PM  management  of 
technical  matters,  schedule,  cost 
schedule/control  and  contract 
administration  and  for  using  the 
"sleeping  bag"  (close  to  the 
contractor)  method  of  managing 
prime  contractors.  My  response  has 
been  to  task  each  ERIS  Project 
Office  Division  Chief  with  the  full 
responsibility  for  technical,  schedule, 
cost  and  resources  (manpower)  in  his 
assigned  area.  We  now  must  execute 
according  to  plan,  anticipate 
problems,  bring  the  best  brains  in  our 
country  to  bear  on  the  key  issues  and 
dig  to  the  level  of  detail  required  to 
hit  rock.  In  brief,  we  will  get  in 
clo.se,  stay  in  close  as  an  interactive 
team  and  let  our  system  of  checks 
and  balances  indicate  the  area  where 
our  limited  resources  can  best  be 
spent  each  day.  What  is  "our 
standard"?  Let  me  explain  by  saying 
any  team  can  react  to  a  problem  - 
that's  "firehouse  management,"  but 
the  ERIS  Project  must  take  steps  to 
ensure  that  the  program  gets  the  edge 
or  margin  required  for  "proactive" 
leadership." 


This  approach  was  applied  to  the  entire 
ERIS  team.  A  Work  Breakdown  matrix  was 
formulated  which  identified  every  element 
to  three  tracking  managers  -  a  government 
manager,  a  prime  contract  manager  and  a 
support  contract  manager.  Each  element 
manager  was  expected  to  know  and  manage 


the  element  and  be  measured,  evaluated  and 
rewarded  or  re-motivated  relative  to 
performance  by  the  appropriately  tiered 
authority.  The  prime  contract  emphasized 
the  generally  accepted  items  of  technical, 
schedule  and  cost  objectives  along  with 
management  information  such  as  Cost 
Performance  Reporting,  the  Work 
Breakdown  Structure  and  Milestone  Event 
tracking.  It  also  imposed  a  stringent 
program  network  tracking  system  which 
transitioned  into  network  uncertainty 
analysis  and  RECOVERY  PLAN/ 
workaround  considerations.  Added  to  the 
bag  of  tools  was: 

•  CONTRACTOR  TOTAL  LIFE 
CYCLE  COST  ESTIMATING 
REQUIREMENT, 

•  low  life  cost  per  kill  goal 
development, 

•  MAKE  OR  BUY  DECISION 
REVIEW, 

•  monthly  status  reviews, 

•  parallel  development  of  high  risk 
hardware  items, 

•  early  trade  studies, 

•  low  cost  booster  development, 

•  SUBCONTRACTOR  REVIEW, 

•  COST  MANAGEMENT 
INCENTIVES. 

A  synopsis  of  those  highlighted  proactive 
techniques  are  presented  in  subsequent 
paragraphs. 


COST  MANAGEMENT  INCENTIVES 

The  contractual  instrument  that  was 
executed,  dramatically  complimented  the 
proactive  style  of  management  called  for  in 
the  PM's  Management  Plan.  It  provided  for 
payment  of  incentive  fee  on  the  basis  of  cost 
performance.  The  contract  established 
target  cost,  target  fee,  and  a  7.‘>;25 
government  to  contractor  share  ratio  for 
overrun/underrun  costs.  Based  on  the  share 
ratio,  a  maximum  potential  fee  was 
established  for  a  low  cost  underrun,  with 
zero  minimum  potential  fee  for  a  high  cost 
overrun  (Figure  1 ). 
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The  contract  provided  for  the  payment  of 
award  fee  based  on  technical  performance, 
including  flight  tests,  and  management 
performance.  Management  performance 
included  but  was  not  limited  to:  planning, 
organizing,  and  controlling  resources: 
balancing  resources,  schedule  and  technical 
perfomiancc  to  achieve  objectives;  reporting 
accurate  and  timely  data  and  cost 
information;  identifying  and  projecting 
solutions  to  potential  and  actual  problems  in 
a  timely  manner;  managing  subcontractors, 
and  providing  general  responsive  program 
support.  The  contract  linked  specific 
milestones  to  award  fee  periods  for  specific 
durations  of  time  and  a  relative  distribution 
of  the  total  award  fee  (Figure  2). 
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If  a  milestone  was  not  accomplished  on 
time,  the  period  remained  open  until  it  was 
accomplished.  The  fee  not  earned  for  each 
period  was  lost  for  succeeding  periods; 
however,  it  rolled  over  to  a  pool  which 
could,  at  the  discretion  of  the  fee 
determining  official,  still  be  received  at  the 
end  of  the  contract.  The  evaluation  process 
was  disciplined  and  formal.  The  contractor 
provided  a  written  self  assessment  covering 
accomplishments  within  ten  work  days  after 
the  end  of  the  period.  Award  fee 
determinations  were  not  open  to  formal 
dispute.  The  motivational  effectiveness  of 
this  tool  cannot  be  understated.  The  prime 
and  subcontractor  team  demonstrated  the 
value  of  the  potential  benefit  offered  via  the 
award  fee,  by  insisting  that  each  of  their 
employees  be  made  aware  of  their  personal 
contribution  by  correlation  to  career 
performance  measurement. 

The  level  of  motivational  intensity  was 
further  raised  by  the  COST 
MANAGEMENT  INCENTIVE  clause  in 
the  contract.  It  provided  that  if  actual 
contract  cost  exceeded  the  high  cost 
estimate,  the  point  at  which  incentive  fee 
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was  already  reduced  to  zero,  then  the 
government  could  provide  additional  funds 
to  continue  the  effort  and  invoke  the  clause. 
It  provided  for  application  of  a  cost  share 
ratio  and  withdrawal  of  Previously  Earned 
Award  Fee  (PEAF).  See  Figure  1.  Specific 
application  addressed  several  tiers  of 
overrun; 

•  For  costs  greater  than  the 
HIGH  COST  estimate  and  not  more  than 
one  hundred  twenty  percent  of  TARGET 
COST,  the  contractor's  share  would  be 
ninety  percent  of  the  overrun  until  one  third 
of  the  PEAF  was  consumed.  If  enough 
PEAF  was  not  available,  the  contractor's 
share  deficit  would  carry  over  to  the  next 
award  fee  period. 

•  For  costs  greater  than  one 
hundred  twenty  percent  but  less  than  one 
hundred  twenty  four  percent  of  TARGET 
COST,  the  contractor's  share  would  be 
eighty  percent  of  cost  until  PEAF  was 
consumed. 

•  For  costs  greater  than  one 
hundred  twenty  four  percent  of  TARGET 
COST,  award  fee  would  be  zero  and  all 
PEAF  not  previously  consumed  would  be 
retunied. 

Points  to  remember  are  that  the  award  fee 
must  be  large  enough  to  attract  attention; 
one  method  of  evaluation  should  be  fair  to 
both  parties;  the  evaluation  criteria  should 
be  understood  and  promulgated  from  the 
beginning;  and  that  recovery  is  possible. 
The  arrangement  should  benefit  both  parties. 
It  is  not  something  the  contractor  should 
assume  is  an  absolute  given,  regardless  of 
performance,  nor  should  the  government 
feel  a  compulsion  to  withhold,  regardless  of 
performance.  The  intent  is  that  it  be  a  win  - 
win  situation  for  both  parties  that  is  entered 
into  in  good  faith.  It  is  a  strong  motivational 
tool.  If  the  dream  of  a  potential  award  fee 
can  motivate,  and  the  mere  thought  of  not 
receiving  the  maximum  available  fee  can 
rededicate  management  attention,  then  the 
nightmare  of  giving  back  all  or  part  of 
previously  earned  award  fee  will  invoke 
absolute  panic. 


SUBCONTRACTOR  REVIEW 

The  contract  established  a  subcontract 
review  cycle  that  involved  project  office 
participation.  The  requirement  did  not  shift 
or  diminish  the  prime  contractor's 
management  responsibility  but  enhanced  the 
government's  position  in  the  loop.  The 
government  received  all  subcontract 
solicitations  whose  estimated  cost  exceeded 
$4M,  as  well  as  certain  other  key 
solicitations  falling  under  that  threshold 
which  were  of  special  interest.  Each 
submission  included  sole  source  justification 
or  evaluation  criteria.  Fifteen  days  prior  to 
the  award  of  a  subcontract  covered  by  the 
clause,  the  prime  contractor  furnished  the 
Procuring  Contracting  Officer 
documentation  supporting  the  recommended 
selection.  The  subcontract  could  not  be 
awarded  without  government  consent  which 
was  required  within  the  fifteen  day  period. 

Other  independent,  yet  interrelated, 
contractual  requirements  were  centered  on 
the  subcontract  process.  Trade  studies  were 
required  during  the  early  months  of  the 
contract.  A  Recompetition  of  Subcontracts 
clause  required  that  as  a  result  of  the  trade 
studies,  recompetition  of  prearranged 
agreements  for  certain  components, 
including  seekers,  might  be  required.  It 
further  stipulated  that  the  requirement  for 
recompetition  would  not  be  a  cause  for 
adjustment  to  the  contract  amounts, 
performance  schedule  or  other  contract 
provisions.  A  Parallel  Technology 
Development  clause  required  that,  as  a  result 
of  the  trade  studies,  certain  parallel 
technology  component  development  might 
be  required.  The  seeker  component  was 
active  in  all  facets  of  the  process  -  the 
subcontract  review,  trade  studies, 
recompetition  of  subcontracts  and  parallel 
development.  Since  the  seeker  was 
considered  to  be  one  of  the  highest 
technical,  schedule  and  cost  risk  items  in  the 
program,  it  was  selected  as  a  risk  mitigating 
dual  source  item.  The  downselect  decision, 
scheduled  for  month  forty-four,  again 
invoked  the  subcontractor  review  process. 
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MAKE  OR  BLY  DECISION  REVIEW 


The  contractor  was  required  to  provide  the 
rationale  and  quotes  supporting  all  make  or 
buy  decisions  for  hardware  WBS  elements 
level  three  and  below,  which  were  estimated 
to  cost  more  than  $3M.  The  internal 
methods  and  procedures  for  determining 
make  or  buy  decisions  were  identified  in  a 
make  or  buy  plan  submitted  with  the 
proposal  and  accepted  at  contract  award. 
The  buy  alternative  had  to  be  supported  by  a 
description  of  competitive  actions  or  sole 
source  justification.  Decisions  were 
presented  to  the  government  for  consent 
with  responses  to  be  returned  within  fifteen 
days.  The  process  was  successful,  stressing 
the  importance  of  economic  trade-offs  in  the 
relationship  to  sound  business  base 
decisions. 


8 


H 


QUANTITY 


A  cost  relationship  to  address  "make**  and  "buy**: 


Where:  FC  =  Fixed  Cost 

A  =  Theoretical  1st  Unit  Cost 
X  =  Quantity 
S  =  Slope 


The  project  office  plans  to  enhance  its  future 
efforts  in  this  area.  The  standard  questions 
relating  to  quantity,  quality,  manufacturing, 
supply,  and  cost  that  affect  make  or  buy 
decisions  are  addressed;  however,  a 
concentrated  effort  will  be  made  to  model  a 
projected  break-even  (BE)  point  for  make  or 
buy  analysis.  One  methodology  being 
considered  is  the  bisection  algorithm  from 
the  field  of  numerical  methods  which 
address  non-linear  models.  Eleven  separate 
cost  relationships  may  be  evaluated  in  the 
methodology.  The  model  addresses 
fixed/start-up  cost  and  variable  cost,  forcing 
the  thought  process  of  Marginal  Theory  and 
Economy  of  Scale.  Figure  3  reflects  a 
potential  output  and  cost  relationship.  The 
BE  point  is  just  one  of  the  variables  which 
will  be  addressed.  Other  evaluating  points 
include  business  base  expansion,  ability  to 
produce,  desire  to  enter  into  a  particular 
environment,  quality  of  product,  importance 
of  item,  etc. 


Figure  3  -  A  Non-Linear  Make  or  Buy  Model 


CONTRACTOR  TOTAL  LIFE  CYCLE 
COST  ESTIMATING  REQUIREMENT 

Early  in  the  program,  the  PM  challenged  the 
government  System  Engineering  Division  to 
field  a  configuration  that  was  cost  effective 
in  total  life  cycle  cost  and  cost  per  kill. 
Rather  than  "arm  wave"  at  the  concept, 
specific  actions  were  initiated  to  move  in 
that  direction.  The  prime  contractor's 
System  Engineering  office  was  established 
as  a  mirror  image  of  its  government 
counterpart.  An  Operations  Researeh 
Analyst,  dedicated  to  cost  analysis  and 
dealing  with  life  cycle  eost  issues,  was 
added  to  that  organization.  The  eontractor 
was  required  to  develop,  deliver  and 
periodically  update,  a  fully  documented  life 
cycle  cost  estimate.  A  government  designed 
PC  model  was  utilized  facilitating  data 
interchange.  The  estimate  eonformed  to 
Army  Cost  Analysis  regulations  and 
proc^ures,  as  did  the  project  office  Baseline 
Cost  Estimate  (BCE).  The  estimates  were 
compared,  tracked  and  evaluated  by  the 
project  office  in  a  manner  similar  to  the 
Independent  Cost  Estimate  -  BCE  track. 
The  requirement  focused  the  eontractor's 
attention  on  the  idea  of  total  life  cycle  cost 
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and  involvement  beyond  the  instant  contract. 
It  provided  an  appreciation  for  long  range 
government  planning  and  prognostication, 
enhancing  overall  support  for  the  acquisition 
process.  An  added  benefit  was  that  cost 
modeling  was  utilized  by  the  contractor  to 
evaluate  early  design  trade  studies  that  not 
only  affected  developmental  hardware  but 
assessed  impact  on  subsequent  production 
and  operational  phases.  Cost  awareness 
permeated  the  contractor’s  functiona'  offices 
and  projected  the  Economic  Man  Tlieory  to 
the  forefront. 


RECOVERY  PLAN 

The  Project  Manager's  Internal  Status 
Review,  a  structured  process  (Figure  4), 
focused  management  attention  and 
corrective  action  toward  problems,  potential 
problems,  trends  or  special  items  of  interest. 

The  review,  conducted  monthly,  was 
presented  to  the  PM  and  Deputy  PM  by 
project  office  Division  Chiefs  and  WBS 
government  managers.  The  briefing  format 
included  a  number  of  standard  charts 
supplemented  by  special  charts.  The 


briefing  started  with  an  overview  of  the  top 
level  summary  of  the  Cost  Performance 
Report  (CPR).  Next  came  a  three  section 
SNAPSHOT  OVERVIEW  chan  (Figure  5) 
with  the  first  section  reflecting  the  status  of 
funds  received,  obligated  and  expended. 
The  second  section  was  a  level  one  CPR 
summary  of  three  months  history  and  six 
months  projection  for  both  cost  and  schedule 
variances.  The  third  section  was  the  project 
manager’s  top  areas  of  concern.  'The 
concern  items  were  individually  identified 
by  title  and  date  of  first  appearance  on  the 
PM’s  list  and  expected  resolution  date.  A 
Management  Reserve  curve  status  chart 
tracked  actual  application  against  the 
cumulatively  diminishing  goal  of  CPR 
management  reserve  measured  at  specific 
milestones  on  a  time  line.  The  intent  was  to 
have  a  specified  amount  of  MR  available  as 
the  system  entered  each  successive  period. 
A  Milestone  Status  chart  tracked  cumulative 
milestones  scheduled  and  completed  and  the 
status  of  delinquencies.  The  segment 
devoted  to  DIVISION  CHIEF  STATUS 
AND  CONCERNS  and  WBS  MANAGER 
PRESENTATIONS  contained  standard 
charts  relative  to  lower  level  WBS  elements 
of  the  CPR,  network  and  schedule. 
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Figure  5  •  Snapshot  Overview 


Also  included  were  uniquely  designed  charts 
and  the  required  RECOVERY  PLAN  charts. 
The  Recovery  Plan  structure  was  an  inte^al 
part  of  the  process.  It  was  used  as  a  vehicle 
to  obtain,  focus  and  maintain  management 
attention  on  cost,  schedule  and  technical 
status  and  corrective  actions  related  to  a 
specific  problem.  The  format  (Figure  6)  was 
specific,  yet  broad  enough,  to  cover  most  of 
the  elements  in  this  section  of  the  briefing. 
The  Recovery  Plan  chart  was  divided  into 
different  sections  and  was  limited  to  one  or 
two  pages  with  associated  backup  charts 
being  acceptable.  The  top  section  of  the 
chart  included  the  identification  control 
number,  date  first  reported,  title,  date  of 
projected  decision  or  resolution  and  date  of 
briefing.  The  next  section  of  the  chart  was 
devoted  to  a  pictorial,  graphic  or  visual 
display  of  the  topic,  problem  and 
recovery/workaround  solution.  The  bottom 
part  of  the  chart  contained  a  more  detailed 
word  display  covering  the  issue,  impact, 
.solution,  key  events,  status  and  responsible 
WBS  Manager. 


trend  movement,  the  network  and  milestone 
tracking  system,  assigned  topics  from  the 
PM,  key  events  or  activities  (e.g..  Critical 
Design  Review),  PM  briefings  and  meetings 
involving  the  project  office  team,  WBS 
manager  presentations.  Division  Chief  status 
presentations,  the  prime  contractor’s  monthly 
In-Process  Review  (IPR)  and  the  Recovery 
Plan/Concern  Log.  Specific  actions 
resulting  from  the  monthly  briefing  were 
recorded  and  tracked  in  the  RECOVERY 
PLAN/CONCERNS  LOG  and  were  added 
or  removed  at  the  PM's  direction.  Items 
tracked  by  the  process  ranged  from  Housing 
at  the  Kwajalein  Atoll  site  to  a  potential 
flight  delay  associated  with  the  air  vehicle 
destruct  initiator.  An  actual  chart  used 
during  that  time,  depicting  the  computer 
module  recovery  plan,  is  shown  at  Figure 
6(A).  The  recovery  plan  concept  was 
adopted  as  an  internal  management  tool  by 
the  prime  contractor  and  often  utilized  in  the 
monthly  IPR  presented  to  the  government. 


The  agenda  and  briefing  elements  evolved 
from  CPR  threshold  breaches  or  significant 
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1.  (Date  First  Reported) 


(Copy  Issue  Statement  from  Snapshot  Chart  • 


(Current  Date  of 
Briefing) 


This  space  is  used  for  a  chart,  graph,  or  picture  that  summarizes  the 
problem,  solution,  status  and  impact. 


Note:  Status  is  new  each  period. 


ISSUE:  An  expansion  of  the  issue  statement  to  provide  perspective  by  adding  specifics  in  quantified  terms  where 

possible. 

IMPACT:  The  engineer  or  manager  responsible  provides  his  best  estimate  -  may  be  an  educated  guess  •  as  to  the 

cost,  schedule,  performance  and  other  impacts. 

SOLLTIQV:  What  must  be  done  to  resolve  the  issue  successfully  -  A  logic,  strategy  or  specific  tasks  are  appropriate  • 

quantified  if  possible. 


KEY  EVENTS:  Key  event  dates  may  be  added  to  solution  >  the  final  decision  date  or  resolution  date  here.  Between 

solution  and  key  events,  the  indicatcH^  and  Uieir  dates  shall  be  made  clear. 

STATUS:  Lang  the  above  indicators,  provide  a  status  as  of  the  submitted  date.  This  may  be  marked  above  and 

noted  as  status.  The  objective  is  to  feel  the  pulse  of  the  manager/engineer  vs.  adhere  to  a  rigid  formaL 

RESPONSIBLE:  Individual  in  project  -  Title  -  Telephone  Number  -  Office  Responsible 


Figure  6  -  Recovery  Plan 


KKLl ABILITY  OF  CONTROL  DATA  CORP  AN/A YK.14 COMPUTER  MODULES  (Auaim  87) 


ISSUE; 


Reiiabitily  of  Block  IT  Modules  is  QuesUooaNe. 
Schedule  for  Block  O  Delivenes  has  no  Slack. 


LMPACT:  Procure  Block  HI  Modules  •  $].85A  Retain  Block  D  Modules  as  Backup  •  $0.5M. 

Delay  of  ERLS  Proaram  •  S6M  P^  Montb 


SOLUTION: 


KEY  fcVE.MS; 


A)  Procure  Block  m  Modules  as  Preferred  Oplioo. 

B)  Retain  Procurement  of  Block  n  Modulei  as  Backup. 

C)  Develop  Workaround  for  Initial  ISTV  Activity. 

A)  Block  ni  Procurement  Dedskm  Jun  87 

B)  Block  II  Buy-Out  Decisioa  Aug  87 

C)  Block  ni  Verification  Tests  Coaplele  Oct  87 

D)  Block  ni  ReliaMlity  Oemonstratioo  Complete  Dec  87 

E)  Decision  to  Qualify  Block  O  for  ERIS  Jan  88 

10  Deliver  First  Block  m  to  Hooeyweii  Apr  88 


STATUS:  Go  Ahead  for  ERIS  Block  ID  Production  on  10  Jun  87. 

RESPONSIBLE:  John  Wymer  -  Avoinics  Project  Engineer  -  895-4#IS 


Figure  6(A)  -  Computer  Module  Recovery  Plan 
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The  Internal  Status  Review,  in  order  to  be 
effective,  must  be  open  to  new  topics  and 
procedures.  As  part  of  this  process,  the 
project  office  plans  to  expand  the  subject  of 
risk  management.  MIL-STD-499B,  Systems 
Engineering  Management,  outlines  an  idea 
that  focuses  on  Technical  Performance 
Measurement  (TPM)  and  encourages  the 
integration  of  TPM  and  Cost  Performance 
Reporting  (CPR)  data.  One  approach  being 
evaluated  is  the  Integrated  Risk  Analysis 
that  addresses  Management  Focus, 
Technical  Performance,  Cost  and  Schedule 
Performance  and  Risk  Exposure.  The  model 
currently  evaluates  CPR,  cost  and  schedule 
variance,  and  multiple  parameter  technical 
performance  status  by  WBS  element.  A 
Management  Focus  chart  (Figure  7)  will 
provide  the  PM  with  a  single  top  level 
summary  measurement  of  the  project's 
overall  health.  The  mathematically  derived 
measure  shows  management  how  focused 
they  are  by  calculating  the  correlation 
between  high  technical  priority,  high 
technical  performance,  and  low  combined 
adverse  cost  and  schedule  variances.  The 
risk  exposure  calculation  ties  together  the 
technical,  cost,  and  schedule  performance 
measures  with  management  focus  and 
provides  a  measure  of  relative  risk  exposure. 
The  measure  ranks  the  selected  WBS 
elements  and  assigns  each  a  risk  weight 
(Figure  8).  The  WBS  elements  with  the 
highest  weights  are  those  that  put  the  project 
at  greatest  risk.  High  risk  exposure  elements 
have  high  technical  importance,  poor 
technical  performance,  and  a  high  combined 
adverse  cost  and  schedule  variance. 


PltHrc  S  •  Relative  Rlifc  bpofurcof  WBS  EkmenU 


CONCLUSIONS/SUMMARY 

The  topics  presented  are  a  small  sample  of 
the  GBI  project's  Integrated  Proactive 
Management  (IPM)  philosophy.  Senior 
project  office  officials  believe  that  the 
process  and  motivational  aspects  are  highly 
significant  and  that  micro  level  specifics  and 
results  follow. 

A  bottom  line  question  must  relate  to  how 
successful  (or,  in  an  imperfect  world,  the 
degree  of  success)  the  1PM  philosophy  was 
in  helping  the  GBI  project  achieve  cost, 
schedule,  and  technical  objectives.  The 
process  of  planning  for  success  began  in 
GBI  prior  to  the  release  of  the  ERIS 
Functional  Technology  Validation  (FTV) 
program  request  for  proposal.  The  actual 
tracking  for  success  started  in  January  1986 
with  the  FTV  contract  award  to  Lockheed 
Missiles  and  Space  Company  (LMSC).  Just 
as  with  most  programs,  ERIS  went  through 
numerous  program  perturbations,  including 
the  San  Francisco  Earthquake  of  1989. 
Throughout  the  entire  period  of 
performance,  the  team  met  each  challenge  in 
a  scientific,  businesslike  manner,  keeping 
the  objective  balanced  with  the  urgency  of 
the  situation  and  economic  impact.  The 
team  would  not  be  pushed  into  panic 
management.  Daily  and  monthly  problems 
were  managed  with  the  end  product  of 
success  in  mind.  The  Project  Manager's 
statement,  "Don't  fail  on  time"  had  a 
leveling  effect. 


el 

8 


> 


Figure  7 .  Mamgemenl  Kocua 


349 


The  areas  of  cost  and  schedule  performance 
were  deemed  highly  successful, 
maneuvering  through  a  maze  of  funding  cuts 
and  uncontrollable  external  adjustments. 
Cost  realism  was  an  early  concern  of  the 
project  office,  as  evidenced  by  an  article 
written  by  a  GBI  employee,  John  E.  Liston. 
The  article  appeared  in  the  September  - 
October  1987  issue  of  the  Program 
Manager,  published  by  the  Defense  Systems 
Management  College  and  was  entitled, 
"Taking  A  Better  Aim  At  Cost  Realism 
Measures."  It  addressed  the  source  selection 
cost  evaluation  process  and  cost  realism  as  it 
related  to  ERIS  FTV.  Included  in  the  article 
was  a  figure  entitled  "Government  Contract 
Risk  Curve"  which  addressed  contract  cost 
as  envisioned  at  that  time.  An  extract  of  the 
curve  is  shown  in  Figure  9,  with,  the  final 
results  superimposed  on  the  original  data. 
The  60  month  contract,  signed  in  January 
1986  for  $466M,  was  completed  17  months 
late  for  $500M.  The  first  flight  against  a 
target,  scheduled  for  June  1990,  was  actually 
flown  in  January  1991,  7  months  late.  It  is 
interesting  to  note  that  while  the  contract 


period  of  performance  grew  28  percent,  the 
cost  delta  was  only  7.3  percent  greater  than 
the  original  contract  price  and  was  near  the 
SSEB  mean  of  $496M.  A  $60M  funding  cut 
early  in  the  program  had  a  ripple  effect  on 
subsequent  schedule  activities.  The  cut 
forced  an  early  selection  of  a  single 
contractor  for  a  risk  mitigating  dual  .source 
seeker  program.  As  a  result,  problems  with 
several  seeker  components  contributed  to  the 
program  delay  and  price  growth. 

The  technical  program  objectives 

established  at  the  very  beginning: 

•  Tactical  design  of  a  low  life  cycle 
cost  midcourse  interceptor 

•  Functionally  emulate  tactical  design 
using  existing  technology 

•  Selecting  a  target  in  the  presence  of 
decoys  and  closely  spaced  objects 

•  Lethality  versus  the  reentry  vehicle 

•  Determining  intercept  pheno¬ 
menology 

were  also  met  with  outstanding  success. 


WF 

X 

496 


Dollars  in  Millions 


Figure  9  -  Government  Contract  Ri.sk  Curve 
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In  summai7,  IPM  as  applied  in  the  GBI 
Project  Office  is  not  intended  to  overmanage 
the  prime  contractor,  support  contractors,  or 
government  team;  however,  it  is  intended  to 
establish  and  maintain  the  proper  degree  of 
presence  and  leverage  throughout  the 
management  and  decision  process.  It 
represents  the  comfort  zone  and  style  that 
this  management  team  views  as  prudent  and 
responsible.  The  functions  of  the  manager 
to  plan,  organize,  direct  and  control,  at  the 
properly  established  level,  is  the  project 
office  dictum. 

An  October  1992  memorandum  from  the 
Army  Acquisition  Executive,  strongly 
endorsed  by  the  GPALS  Program  Executive 
Officer  (PEO),  emphasized  a  Team  Concept 
based  on  the  premise  that  both  government 
and  contractor  share  a  common  goal  in 
management  efforts. 

"Our  vision  of  team  management  is 
pulling  together  all  the  right  people, 
from  government  and  industry,  that 
have  management  authority  and 
responsibility  for  a  program  to  form 
an  accountable  team  that  is  success 
oriented.  We  want  to  get  top  level 
management  involved  earlier,  and 
have  the  right  people  on  the  team  to 
resolve  problems  quickly  with 
minimum  cost  and  time  impact.  In 
other  words,  those  individuals  who 
expect  to  ultimately  "grade  the 
paper"  are  expected  to  help  "write 
the  paper"  up  front.  The  team  will  be 
expected  to  define  critical  processes 
and  identify  management  actions 
required  to  ensure  program  success." 

The  perpetually  evolving  style  of  Integrated 
Proactive  Management  has  worked  for  the 
GBI,  fits  within  the  AAE's  Team  Concept 
and  will  transition  the  Ground  Based 
Element  of  GPALS  to  a  successful  future. 


element  of  the  National  Missile  Defense 
(NMD)  segment  of  the  Global  Protection 
Against  Limited  Strikes  (GPALS)  System. 

James  C.  Katechis  has  served  as  Project 
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two  awards  for  the  Most  Significant 
Achievement  in  Strategic  Defense,  one  from 
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Professional  Engineers;  the  Meritorious 
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ENDNOTES 

The  Ground  Based  Interceptor  (GBI)  Project 
Office  is  a  continuation  of  the 
Exoatmospheric  Reentry  Vehicle  Interceptor 
Subsystem  (ERIS)  project.  The  GBI  is  an 
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ABSTRACT 

This  research  was  conducted  to:  (1) 
examine  the  history  of  the  weapon  system 
warranty  (WSW)  requirement;  (2)  assess 
the  relevancy  of  warranties  to  dormant 
weapon  systems;  (3)  review  the  current 
practices  concerning  the  performance  of  a 
cost-benefit  analysis  (CBA)  to  support  the 
waiver  of  a  warranty  for  a  dormant  weap¬ 
on  system;  and  (4)  identify  factors  that 
affect  the  warranty  cost-benefit  analysis 
(CBA)  for  a  dormant  weapon  system. 

The  Defense  Federal  Acquisition 
Regulation  Supplement  (DFARS)  and  Air 
Force  regulations  require  that  a  CBA  be 
performed  prior  to  the  purchase  of  a 
warranty.  The  program  manager  makes 
the  warranty  purchase  decision  based 
upon  the  CBA.  If  the  analysis  is  not 
performed  adequately,  the  decision  may 
be  based  on  erroneous  results;  therefore, 
CBAs  must  be  accurate. 

Dormant  weapon  systems  are  a 
unique  category:  they  are  non-operating, 
non-reusable,  and  non-restorable.  The 
systems  are  destroyed  upon  use  in  the 
field,  and  that  fact  poses  serious  difficul¬ 
ties  in  performing  warranty  CBAs  for  such 
systems.  Some  dormant  weapon  systems 


were  examined,  and  factors  used  in  per¬ 
forming  the  CBA  for  each  program  were 
identified  and  are  reported  in  this  paper 
(See  Figure  1). 

INTRODUCTION 

For  a  major  weapon  system,  the 
cost  of  a  warranty  can  be  a  significant 
portion  of  total  system  cost.  Therefore,  a 
cost  benefit  analysis  (CBA)  which  exam¬ 
ines  the  cost-effectiveness  of  a  proposed 
warranty  is  a  necessary  internal  control 
measure.  Without  an  adequate  CBA,  the 
government  may  spend  funds  on  warran¬ 
ties  that  are  not  cost-effective.  If  the 
analysis  does  not  Justify  the  purchase  of  a 
warranty,  a  waiver  should  be  requested. 
Several  government  organizations  have 
found  deficiencies  in  CBAs,  and  have 
recommended  improvement  of  the  analysis 
process. 

THE  HISTORY  OF  AIR  FORCE 
WARRANTIES 

The  history  of  Air  Force  Warran¬ 
ties  is  rooted  in  the  history  of  Department 
of  Defense  weapon  systems  warranties  in 
general.  The  warranty  provision  was 
mandated  by  public  law  after  Congress 
voted  in  1983  to  require  manufacturers  to 
provide  weapon  system  warranties.  The 
requirement  stemmed  from  the  procure¬ 
ment  "horror  stories"  in  the  early  1980s, 
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and  was  spearheaded  by  Senator  Mark 
Andrews,  a  republican  from  North  Dako¬ 
ta.  [4]  Much  debate  about  the  issue 
occurred  at  the  Armed  Services  Hearing 
in  February  1984,  and  several  witnesses, 
including  Secretary  of  Defense  Richard  D. 
DeLauer,  the  chief  Pentagon  witness, 
spoke  against  the  proposed  law.  [4]  In 
general,  it  is  believed  that  warranties  do 
not  provide  incentives  to  contractors  to 
make  improvements  in  technology  beyond 
what  is  required  by  the  warranty.  This 
sentiment  was  expressed  by  Air  Force  Lt. 
Gen.  James  W.  Stansberry  who  warned 
the  panel,  "Once  modest,  unchallenging 
[performance]  goals  are  set,  they  tend  to 
become  self-fulfilling  prophecies".  [4] 

Contractors  presented  several 
arguments  against  the  proposed  law:  The 
Department  of  Defense,  as  well  as  the 
contractor,  controls  the  design  of  a  system; 
therefore,  the  government  shares  respon¬ 
sibility  with  the  contractor  in  this  area.  [4] 
Warranties  discourage  innovations  and 
breakthroughs  in  technology.  [4]  Critical 
differences  exist  between  commercial  and 
military  equipment,  and  these  differences 
were  not  being  considered.  [2] 

Still,  Sen.  Andrews  was  unrelenting 
in  his  efforts.  Congressional  interest  re¬ 
mained  high,  and  the  public  law  was 
passed  as  part  of  the  1984  Defense  Appro¬ 
priation  Act.  The  law  requires  each  prime 
contractor  for  the  production  of  a  weapon 
system  to  provide  written  guarantees, 
starting  with  procurements  after  1  January 
1985.  [2]  The  three  types  of  guarantee 
covered  by  the  law  are  1)  design  and 
manufacturing,  2)  materials  and  workman¬ 
ship,  and  3)  performance  specifications. 


1.  Federal  Acquisition  Regu¬ 
lation  (FAR),  Subpart  46.7, 

FAC  84-58, 

2.  Defense  Federal  Acquisition 
Regulation  Supplement 
(DFARS),  Subpart  46.770-8, 

DAC  88-15, 

3.  Air  Force  Regulation 
(APR)  173-15,  Cost  Analy¬ 
sis.  4  Mar  88,  and 

4.  Air  Force  Regulation 
(AFR)  70-11,  Acquisition 

Management: _ Weapon 

System  Warranties.  1  Dec 

88. 

The  public  law  is  contained  in  10  United 
States  Code  (USC)  2403. 

THE  COST  BENEFIT  ANALYSIS  (CBA) 

Affordability  must  be  considered  in 
relation  to  potential  benefits  derived  from 
the  warranty.  AFR  70-11  contains  guid¬ 
ance  for  implementing  DOD  policy  on 
weapon  system  warranties  in  the  Air 
Force,  which  policy  requires  that  only 
cost-effective  warranties  be  obtained. 
This  means  that  a  cost  benefit  analysis 
(CBA)  must  always  be  performed  to  ascer¬ 
tain  whether  or  not  a  warranty  would  be 
cost  effective  for  a  given  acquisition. 
Even  if  the  contractor  proposes  a  "no-cost" 
warranty,  a  CBA  must  be  performed  to 
compare  the  government’s  cost  of  adminis¬ 
tering  and  enforcing  the  warranty  to  the 
potential  benefits  to  be  derived  from  the 
warranty.  [6] 

The  literature  on  this  topic  contin¬ 
ues  to  address  the  shortage  of  guidance  on 
how  to  prepare  warranty  CBAs,  as  well  as 
the  nonspecific  nature  of  the  guidance  that 
does  exist.  There  is  a  strongly  felt  need 
for  valid  CBA  procedures. 
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The  RAND  Corporation  published 
a  report  in  1987  regarding  weapon  system 
warranties.  The  report,  which  attempted 
to  answer  the  question,  "Can  warranties 
help  in  improving  weapon  cost,  schedule, 
performance,  and  reliability?"  reviewed 
implementation  and  theory  of  warranties. 
[10]  The  authors  were  concerned  that 
DOD  was  spending  funds  on  negotiating, 
implementing,  and  enforcing  warranties 
with  no  knowledge  of  the  cost-effective¬ 
ness.  The  report  found  that  ".  .  .there 
[was]  a  real  need  for  improved  evaluation 
criteria  and  procedures."  The  report  also 
expressed  concern  that  waiver  requests 
were  not  being  submitted,  and  noted  the 
need  for  more  policy  in  this  area.  This 
strongly  suggests  that  procedures  may 
need  to  be  improved  and  that  a  compre¬ 
hensive  review  process  may  need  to  be 
implemented.  [10] 

Another  RAND  Corporation  report 
on  warranties,  titled  A  New  View  of 
Weapon  System  Reliability  and  Maintain¬ 
ability.  was  published  in  early  1989.  The 
report  was  primarily  directed  towards  a 
review  of  reliability  and  maintainability 
weaknesses  in  Air  Force  systems,  but  it 
also  addressed  the  cost  of  warranties. 
Further,  the  report  took  the  position  that 
there  should  be  evidence  of  a  connection 
between  use  of  warranties  and  increased 
levels  of  reliability  and  maintainability 
prior  to  purchase  of  a  warranty.  As  did 
the  earlier  RAND  report,  the  1989  report 
stated  that  the  services  were  spending 
money  without  knowing  whether  or  not 
benefits  would  be  derived.  Specifically, 
the  report  stated  that  "...data  and  contrac¬ 
tual  language  are  being  gathered  sporadi¬ 
cally  by  various  agencies...,"  and  went  on 
to  conclude  that  no  process  for  systemati¬ 
cally  gathering  and  coordinating  informa¬ 
tion  existed.  [7]  Warranty  effectiveness 
cannot  be  evaluated  if  the  essential  data 
cannot  be  gathered. 


In  September  1989,  following 
audits  of  three  procurement  activities,  the 
United  States  General  Accounting  Office 
(GAO)  published  a  report  to  the  Secretary 
of  Defense  on  DOD  warranties.  One  pur¬ 
pose  of  the  audits  was  to  determine  if 
cost-effectiveness  analyses  were  being 
performed  as  required  by  both  DOD  and 
service  regulations.  [11]  The  GAO  discov¬ 
ered  that  CBAs  were  either  not  being 
performed  or  were  not  being  performed 
adequately,  and  therefore,  did  not  support 
a  valid  conclusion  that  the  proposed  war¬ 
ranty  was  cost-effective.  The  report  also 
stated  that  the  services  were  not  consider¬ 
ing  waivers  as  an  option  during  the  pro¬ 
curement  process  [11]. 

An  Inspector  General  report  titled. 
System  Acquisition  Management  (SAMI') 
of  Weapon  System  Warranties,  dated  15 
August  1990,  reviewed  several  aspects  of 
warranties,  one  of  which  was  the  CBA. 
The  team  found  that  none  of  the  organi¬ 
zations  reviewed  had  implemented  a  good 
CBA  process,  noting  deficiencies  in  both 
the  review  process  and  the  supporting  data 
base.  The  report  further  suggested  that 
deficiencies  in  the  data  base  may  render 
many  CBAs  invalid.  The  report  recom¬ 
mended  that  action  be  taken  to  "Devise 
and  build  a  data  base  of  essential  cost 
parameters  and  relationships  to  support 
effective  warranty  CBAs."  [5] 

Guidance  for  Performing  CBAs.  In 
1991,  the  Air  Force  Communications 
Command  at  Scott  Air  Force  Base  pub¬ 
lished  a  Warranty  Guide,  which  provides 
general  information  on  warranties.  A  sec¬ 
tion  titled,  "What  Should  a  Warranty 
Cost?"  notes  that  "[i]t’s  difficult  to  say  ex¬ 
actly  what  a  warranty  should  cost".  [1] 
The  guide  then  suggests  that  the  analyst 
should  first  ask  the  contractor  what  is 
covered,  and  also  recommends  the  use  of 
a  life-cycle  cost  model.  From  there  the 
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guide  lists  very  general  questions  pertain¬ 
ing  to  potential  costs  and  benefits  that 
should  be  answered  when  using  the  model. 
No  specific  guidelines  for  performing  the 
CBA  are  provided;  the  limited  content  of 
this  warranty  guide  illustrates  the  need  for 
stronger  guidance  for  performing  CBAs. 

Writing  in  the  Program  Manager, 
and  referring  specifically  to  sole-source 
and  cost-type  acquisitions.  Brigadier  Gen¬ 
eral  Lewis  E.  Curtis  said  that  weapon 
system  warranties  do  not  make  sense  in 
every  procurement  situation.  He  further 
stated  that  unless  a  warranty  can  clearly 
be  shown  to  be  beneficial,  it  makes  good 
business  sense  to  pursue  a  waiver.  Gener¬ 
al  Curtis’s  position  is  especially  relevant  in 
the  case  of  dormant  weapon  systems,  be¬ 
cause  the  nature  of  such  systems  is  such 
that  it  is  difficult,  if  not  impossible,  to 
show  that  a  warranty  is  clearly  beneficial. 
The  authors  think  that  it  is  in  connection 
with  this  type  of  weapon  system  that  guid¬ 
ance  on  the  preparation  of  a  valid  CBA  is 
most  sorely  needed.  This  is  e.specially  true 
with  respect  to  the  essential  performance 
requirements  (EPR)  warranty,  a  concept 
which  will  be  di.scus.sed  below. 

THE  ESSENTIAL  PERFORMANCE 

REQUIREMENTS  WARRANTY 

A  .system’s  EPR  warranty  typically 
drives  the  major  portion  of  warranty  cost. 
As  a  weapon  system  moves  away  from  the 
concept  exploration  stage,  the  likelihood 
of  influencing  design  to  affect  EPRs  de¬ 
creases.  Once  a  system  is  in  the  engineer¬ 
ing  and  manufacturing  development  stage, 
it  is  very  co.stly  for  the  contractor  to  im¬ 
prove  essential  system  performance, 
whether  or  not  it  meets  the  requirements 
and  the  customer’s  expectations.  When 
deficiencies  are  detected,  whether  through 
operational  tests  and  evaluations  (OT&E) 
or  during  the  course  of  normal  use  after 


the  system  is  fielded,  warranty  claims  can 
be  made.  However,  remedies  vary  with 
different  contracts;,  and  a  test  failure  may 
not  prompt  the  contractor  to  redesign  a 
system  to  meet  the  EPR,  even  if  it  is  clear 
that  the  failure  is  due  to  a  faulty  design. 
The  contractor’s  analysis  may  indicate  that 
it  is  more  cost  effective  (from  the  perspec¬ 
tive  of  the  contractor)  to  pay  the  remedies 
for  breaching  the  EPR  warranty  than  to 
make  an  engineering  change  to  the  .system 
to  meet  the  EPR.  Therefore,  the  warranty 
contract  should  specifically  require  rede¬ 
sign  of  a  deficient  .system. 

TTie  design  of  a  system  and  the 
manufacturing  processes  employed  have  a 
direct  relationship  to  the  EPR  warranty. 
Reliability  is  designed  into  the  system 
during  the  development  stages.  Manu¬ 
facturing  control  processes  can  improve  a 
system’s  reliability  by  detecting  faulty 
workmanship  and  an  inadequate  manufac¬ 
turing  process.  The  manufacturing  process 
can  then  be  improved,  resulting  in  a  high¬ 
er  level  of  reliability.  If  EPRs  are  satisfac¬ 
torily  met,  then  it  is  likely  that  the  other 
parts  of  the  warranty  would  be  positively 
affected. 

EPR  Selection.  Operational  per¬ 
formance  requirements,  which  are  derived 
from  the  Operational  Requirements  Docu¬ 
ment  (ORD),  form  the  basis  for  the  EPR 
warranty.  The  ORD  is  prepared  by  the 
user  or  the  user’s  representative.  [6]  The 
requirements  for  the  EPR  should  be  .se¬ 
lected  based  on  attributes  that  cannot  be 
determined  with  certainty  prior  to  or 
during  acceptance  testing.  Verification  of 
EPRs  occurs  during  the  operations  phase 
after  the  system  is  fielded.  EPRs  should 
be  measurable  and  verifiable  in  order  to 
make  valid  claims  and  to  perform  valid 
CBAs.  Availability,  mission  capability, 
and  turn-around  time  are  examples  of 
potential  EPRs.  Clear  and  conci.se  re- 
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quirements  and  measurements  must  be 
stated  in  the  warranty.  For  example,  a 
warranty  may  state  the  requirement  for  a 
mean-time-between-maintenance  of  250 
hours. 

Essential  Performance  Parameters 
(EPFs').  EPPs  for  a  typical  weapon  system 
(for  example,  an  aircraft  engine  that  accu¬ 
mulates  flying  hours  on  a  steady  basis)  are 
monitored  continuously  as  the  system 
performs  in  the  field.  On  the  other  hand, 
EPPs  for  dormant  weapon  systems  are 
difficult  to  monitor  because  they  are  not 
operated  in  field  conditions  on  a  regular 
basis.  A  dormant  system  usually  under¬ 
goes  a  pre-acceptance  test  prior  to  deliv¬ 
ery  to  the  government.  Following  delivery 
the  system  is  stored,  and  at  some  specified 
point  in  time,  the  depot  may  perform 
serv'ice  maintenance  on  the  system.  Ser¬ 
vice  maintenance  may  include  replacing 
some  components,  or  performing  an  exter¬ 
nal  check  for  rust  and  humidity  damage;  it 
can  include  both  replacement  and  external 
checks.  The  system  may  be  tested  at 
specified  intervals,  determined  by  the 
warranty  contract,  to  verify  the  EPPs 
against  the  EPRs.  Destructive  testing  is 
the  only  test  option  for  some  .systems, 
while  test  sets  can  be  used  for  other  .sys¬ 
tems. 

APPLICATION  OF  EPRS  FOR 

SPECIFIC  DORMANT  WEAPON 

SYSTEMS 

The  EPR  warranties  for  the  Peacekeeper 
Missile  Propulsion  System,  Advanced 
Cruise  Missile  (ACM),  ACM  Propulsion 
System,  and  Maverick  Missile  were  re¬ 
viewed.  The  following  information  was 
obtained  for  each  system:  1)  specific 
EPRs  warranted,  2)  type  of  tests  per¬ 
formed  to  determine  if  a  breach  in  EPR 
warranty  has  occurred,  and  3)  program 
office  comments  about  the  EPR  warranty. 


Peacekeeper  Missile  Propulsion 
System.  The  specific  EPRs  covered  under 
the  warranty  were:  action  time,  total 
impulse,  instantaneous  thrust,  thrust  tail- 
off,  ignition  delay,  thrust  vector  actuator 
(TVA)  enable  time,  TVA  action  time, 
TVA  slew  rate,  TVA  duty  cycle,  stage  IV 
duty  cycle.  The  test  performed  was  opera¬ 
tional  test  and  evaluation  (OT&E). 
OT&E  included  random  .selection  and 
testing  of  deployed  missiles  from  those  on 
alert  at  operational  miissile  wings.  OT&E 
was  u.sed  to  verify  the  actual  performance 
reliability  of  the  operational  missile  fleet. 
A  waiver  to  the  EPR  warranty  was  grant¬ 
ed  in  1988  based  on  non-cost  effective¬ 
ness;  the  waiver  covered  the  warranties  for 
the  Stage  IV  production  contract  and  be¬ 
yond. 

The  waiver  application  reflected  a 
concern  about  having  "little  or  no  oppor¬ 
tunity  to  determine  if  the  missile  would 
perform"  successfully,  and,  further,  pointed 
up  the  futility  of  attempting  to  identify  the 
cau.se  of  a  failure,  should  one  occur.  [3] 

Advanced  Cruise  Missile  (ACM). 
The  ACM  has  been  in  limited  production 
for  three  years.  The  specific  EPR  war¬ 
ranted  is  stated  as  percentage  of  systems 
available.  Determination  of  a  breach  of 
warranty  is  based  on  assessed  distribution 
of  the  mi.ssile’s  reliability  during  ground 
tests.  The  specific  tests  are  loaded 
launcher  pylon  test  (LEFT)  and  .system 
integration  test  (SIT). 

The  facts  concerning  this  .system 
are  similar  to  those  pertaining  to  the 
Peacekeeper  Missile  Propulsion  System. 
It  has  been  found  that  flights  of  the  ACM 
are  "too  infrequent"  to  be  of  much  value 
in  enforcing  a  breach  of  warranty.  There¬ 
fore  the  ACM  System  Program  Office 
intends  to  submit  a  total  warranty  waiver 
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application  for  the  latest  negotiated  lot 
buy  based  on  non-cost  effectiveness.  [8] 

Advanced  Cruise  Missile  Propul¬ 
sion  System.  The  specific  EPR  warranted 
was  defined  as  a  failure  rate.  However,  a 
full  warranty  waiver  was  granted  for  the 
.system  in  March  1991.  At  that  time  the 
system  was  in  the  production  lot  II  buy. 
The  waiver  was  justified  on  the  basis  that 
the  acceptance  testing  procedures  are 
.sufficient  to  ensure  the  quality  of  the  units 
received.  In  addition,  the  waiver  applica¬ 
tion  addressed  the  fact  that  the  dormant 
nature  of  the  system  posed  a  difficulty  in 
enforcing  a  warranty  breach.  [12] 

Maverick  Missile  System.  The 
specific  EPR  warranty  states  that  ninety- 
sw  percent  of  inspected  missiles  shall  pass 
an  inspection  cycle.  Visual  and  functional 
tests  are  performed  on  approved  test  sets. 
The  missile  has  been  in  production  for 
more  than  twenty  years,  and,  according  to 
the  warranty  manager,  has  had  an  EPR 
warranty  on  all  units  produced,  except  for 
low-rate  initial  production  lot.  The  war¬ 
rants  is  not  priced  separately;  the  cost  is 
included  in  the  total  missile  cost.  There¬ 
fore,  a  valid  cost-effectiveness  analysis  is 
not  possible  in  this  instance.  It  is  funda¬ 
mental  that  the  costs  and  benefits  of  a 
warranty  must  be  compared  to  each  other 
to  determine  cost-effectiveness. 

TTie  missile  program  has  never  had 
a  breach  of  the  EPR  warranty,  and  the 
Program  Office  is  satisfied  with  the  war¬ 
ranty.  The  contractor  recently  reduced 
the  failure  rate  from  five  percent  to  four 
percent,  an  achievement  that  program 
office  personnel  attribute  to  the  existence 
of  the  EPR  warranty.  Despite  the  fact 
ihat  the  Maverick  Mi.ssile  Program  Office 
is  .satisfied  with  the  warranty,  there  is  no 
way  of  ensuring  that  the  warranty  is  cost 


effective.  Nevertheless,  the  warranty 
manager  revealed  no  plans  to  request  a 
waiver  of  the  warranty.  [9] 

Combined  Effects  Munitions 
(CEMl  System.  The  EPR  warranty  for 
the  CEM  states  that  the  function  rate  of 
the  system  will  not  degrade  below  a  spe¬ 
cific  percentage.  The  function  rate  is 
defined  as  the  number  of  units  (bomblets) 
that  function  from  all  succe.ssful  dispenser 
events  divided  by  the  total  number  of 
units  released  from  all  successful  dispenser 
events  minus  no-test  units.  Triennial 
testing  of  30  systems  per  production  lot 
for  the  10-year  warranty  is  the  basis  for 
measuring  the  function  rate.  The  .system 
was  produced  by  two  contractors  until 
1990.  In  1990  the  system  came  under  a 
sole-source  contract.  To  date,  no  warranty 
claims  have  been  made  against  the  EPR 
warranty.  However,  a  claim  was  made 
against  a  defective  item. 

Two  views  of  warranty  effectiveness 
for  the  CEM  can  be  taken.  First,  be¬ 
cause  the  system  is  performing  satisfactori¬ 
ly  with  no  failures,  the  EPR  warranty  has 
been  effective.  The  fact  that  the  contrac¬ 
tor  has  produced  a  system  that  has  not 
failed  suggests  that  the  warranty  is  work¬ 
ing  as  intended.  Second,  a  warranty  has 
been  purchased,  but  the  government  is  not 
receiving  tangible  benefits  such  as  warran¬ 
ty  repairs;  therefore,  the  warranty  is  not 
cost-effective,  because  the  measurable 
benefits  are  zero.  These  two  possible,  yet 
conflicting,  interpretations  point  up  the 
difficulty  of  justifying  warranties  for  dor¬ 
mant  weapon  .systems. 

DOES  A  GENERAL  METHODOLOGY 
FOR  DORMANT  WEAPON 
SYSTEMS  EXIST? 

The  information  gathered  on  the 
dormant  weapon  .systems  di.scu.ssed  above 
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will  now  be  examined  to  see  how  well  a 
warranty  on  a  dormant  weapon  systems 
relates  to  the  warranty  objectives.  Fur¬ 
ther,  the  information  will  be  examined  to 
determine  whether  or  not  it  suggests  that 
a  general  cost-benefit  analysis  methodolo¬ 
gy  can  be  developed  for  dormant  weapon 
systems.  Following  is  a  review  of  the 
factors  considered  in  performing  the  CBAs 
for  the  Peacekeeper  Missile,  the  ACM, 
the  ACM  Propulsion  System,  the  Maverick 
Missile,  and  the  CEM. 

For  warranties,  the  benefit  derived 
is  the  cost  avoidance  to  the  Government 
that  results  from  having  the  contractor 
perform  repair  and  replacement  actions  or 
correct  problems  at  his  own  expense,  plus 
the  cost  avoidance  from  not  having  to 
m.aintain  an  organic  repair  shop  and  train 
and  pay  repair  technicians.  The  cost  of  a 
warranty  is  that  of  buying  coverage  from 
the  contractor,  plus  the  cost  of  a  system  to 
document,  track,  and  report  claims  against 
the  warranty.  (3| 

Peacekeeper  Missile  CBA.  Factors 
included  in  the  analysis  were  cost  of  ac¬ 
quisition,  cost  of  preparing  or  setting  up 
each  item  (aircraft,  missile,  ship,  truck, 
computer,  etc.)  for  operation,  cost  of 
routine  preventative  maintenance,  cost  of 
failure  repair,  cost  of  spare  parts,  and  the 
cost  of  having  or  contracting  a  repair 
shop,  including  labor.  [.3|  Comparison  of 
life  cycle  cost  with  and  without  warranty 
was  performed. 

Advanced  C  ruisc  Missile  CBA. 
Factors  included  in  the  analysis  were  reli¬ 
ability  assessments,  a  cumulative  availabili¬ 
ty  rate  (('AR),  the  cost  of  warranty  engi¬ 
neering  change  proposals  (FlCPs),  and  the 
cost  of  money.  The  program  office  ap¬ 
plies  statistical  tmalysis,  including  sensitivi¬ 
ty  analysis,  to  [)redict  the  probability  of 
failures  anti  the  consequences  thereof. 


The  last  CBA  performed  indicated  that 
one  of  the  contractor’s  warranties  was  not 
cost-effective,  and  the  program  office  is 
currently  seeking  a  waiver. 

ACM  Propulsion  System  CBA. 
The  CBA  for  the  ACM  propulsion  system 
was  based  on  historical  data  for  the  sys¬ 
tem,  as  well  as  for  the  predecessor  system. 
The  historical  data  consisted  of  the  num¬ 
ber  and  types  of  failures  that  occurred, 
and  the  cost  of  repair  for  those  failures. 
The  data  was  used  to  make  an  engineering 
projection  for  reliability  and  projected 
overall  failure  rates.  A  sensitivity  analysis 
was  performed  comparing  expected  costs 
and  benefits  against  a  worst  case  scenario 
in  which  the  benefits  would  be  lower  than 
expected.  Non-quantifiable  factors  such 
as  missile  deliveries  falling  behind  .sched¬ 
ule  were  included  in  the  narrative.  A 
comparison  of  life  cycle  costs  with  and 
without  the  warranty  was  made.  The  last 
CBA  performed  indicated  the  warranty 
was  not  cost-effective,  and  a  waiver  to  the 
total  warranty  was  obtained  in  March  of 
1991. 

Maverick  Missile  System  CBA. 
The  Maverick  CBA  was  based  on  output 
from  The  Maverick  Cost-of-Ownership 
Model  (MAVCOM).  This  model  calcu¬ 
lates  all  support  life  cycle  costs  (cost  of 
ownership)  associated  with  maintaining 
the  inventory  of  the  Maverick  missile 
throughout  its  operational  life.  The  model 
allows  the  performance  of  sensitivity  anal¬ 
yses  such  as  examining  the  cost  differences 
associated  with  incorporating  an  engineer¬ 
ing  change  into  production,  or  with  chan¬ 
ging  an  operational  or  support  parameter. 

The  SPO  has  never  encountered  a 
breach  of  the  EPR  warranty,  and  the 
warranty  is  not  separately  priced.  It  is 
questionable  that  a  warranty  can  be  identi¬ 
fied  as  cost-effective  if  it  is  not  separately 
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priced.  However,  the  SPO  personnel  feel 
that  the  warranty  has  provided  incentive 
for  the  contractor  to  reduce  the  failure 
rate  from  five  percent  to  four  percent  (the 
contract  threshold).  The  mi.ssile  has  been 
in  existence  for  over  twenty  years. 

Combined  Effects  Munitions 
(CEM^  CBA.  The  CBA  for  the  CEM 
compares  the  government  cost  with  and 
without  a  warranty  for  the  following  fac¬ 
tors:  warranty  price,  warranty  administra¬ 
tion,  test  administration,  conduct  of  test¬ 
ing,  and  ECPs.  The  analysis  indicated 
that  government  costs  were  higher  without 
a  warranty  due  to  the  higher  cost  of  test 
administration  and  higher  cost  of  conduct¬ 
ing  tests.  Therefore,  the  purchase  of  a 
warranty  was  determined  to  be  cost-effec¬ 
tive.  To  date,  no  EPR  warranty  claims 
have  been  made. 

CONCLUSIONS 

A  review  of  similar  dormant  wea¬ 
pon  systems  indicated  that  a  unique  attrib¬ 
ute  of  such  systems  was  the  use  of  destruc¬ 
tive  testing  for  measuring  e.ssential  perfor¬ 
mance  requirements.  The  high  cost  of 
destructive  testing  limited  the  number  of 
tests  performed  to  ascertain  whether  or 
not  a  system’s  performance  requirements 
were  satisfactorily  met.  Two  of  the  .sy.s- 
tems  reviewed  were  granted  warranty 
waivers  on  the  basis  of  a  CBA  indicating 
the  warranty  was  not  cost-effective,  be¬ 
cause  the  dormant  nature  of  the  sy.stem 
limited  testing  opportunities. 

The  CBAs  for  the  dormant  weapon 
systems  reviewed  were  examined,  and  it 
was  found  that  the  general  approach  taken 
for  all  of  the  CBAs  was  the  use  of  histori¬ 
cal  data  and  projections  for  future  costs 
versus  expected  benefits.  Sensitivity  analy¬ 
sis  was  performed  to  varying  degrees. 
I^xccpt  for  the  Maverick  Missile,  all 


analyses  performed  also  included  life  cycle 
cost  with  and  without  a  warranty.  The 
Maverick  Missile  CBA  appears  to  be  the 
weakest  for  the  following  reasons:  1)  it 
does  not  compare  life  cycle  cost  with  and 
without  a  warranty,  2)  the  only  cost-bene¬ 
fit  sensitivity  performed  compares  a  four 
and  five  percent  failure  rate  for  the  EPR 
warranty,  and  3)  the  cost  of  the  warranty 
is  not  known  (cost  is  included  in  the  mis¬ 
sile  price).  The  Maverick  Missile  warran¬ 
ty,  however,  is  the  only  one  that  shares 
risk  with  the  contractor  by  accepting  re¬ 
sponsibility  for  the  first  four  percent  of 
failures.  Risk  sharing  reduces  the  warran¬ 
ty  cost. 

Figure  1  illustrates  that,  for  the 
warranty  CBA,  the  .systems  reviewed  use 
many  different  cost  categories,  and  only  a 
few  categories  are  used  to  measure  bene¬ 
fits.  Intangible  benefits  such  as  increased 
mission  readiness  and  increased  availabili¬ 
ty  were  not  included  in  any  of  the  CBAs. 
The  fact  that  benefits  are  diffwi!'. 
define  tends  to  support  the  GAG  •  fi  < 
ings  that  CBAs  are  inadequate. 

Although  some  similarity  exists 
among  the  CBAs,  the  differences  in  the 
individual  warranties  do  not  suggest  a 
universal  methodology  can  or  should  be 
developed.  Warranties  are  tailored  to 
meet  the  requirements  of  the  specific 
system,  and  no  two  are  exactly  alike.  The 
remedies  specified  in  each  individual 
warranty  determines  the  factors  that 
should  be  used  to  calculate  costs  and 
benefits. 
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Figure  1:  Comparison  of  Cost-Benefit  Analysis  Inputs 
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ABSTRACT 

There  is  a  strong  mandate  within  the 
Federal  Government  for  agency  heads  to 
oversee  development  of  their  agency’s  procure¬ 
ment  systems  and  to  conduct  procurement 
reviews  to  ensure  those  systems  meet  approved 
criteria. 

Currently,  however,  there  is  neither  an 
accepted  uniform  methodology  for  conducting 
reviews  nor  standards  for  measuring  the 
quality  of  an  agency’s  acquisition  performance 
that  has  been  developed  for  Government-wide 
application.  To  meet  this  need,  the  Logistics 
Management  Institute  (LMI)  developed  a 
review  methodology  incorporating  its  “best 
evaluation  practices”  based  on  its  extensive 
prior  review  experience  and  also  inputs  from 
other  practitioners. 

This  paper  first  briefly  describes  some 
historical  methods  and  techniques  used  in 
conducting  acquisition  and  purchasing  reviews 
and  then  addresses  some  innovative  strategies, 
techniques,  methods,  and  applications 
developed  by  the  Logistics  Management 
Institute  (LMI). 

INTRODUCTION 

Managers  in  both  the  private  and  public 
sectors  have  been  vitally  interested  in 
ensuring  effective  purchasing  practices  in 
recognition  of  the  critical  role  that  purchasing 
plays  in  achieving  organizational  objectives. 
Over  the  past  9  years  LMI  has  had  the 
opportunity  to  develop  and  test  an  acquisition 
management  review  methodology  at  a  number 
of  public  agencies.  The  purpose  of  our  reviews 
is  to  independently  assess  the  effectiveness  and 
quality  of  the  agency’s  acquisition  process  and 


to  provide  specific  recommendations  for 
improving  the  acquisition  system.  Our  acqui¬ 
sition  review  begins  with  the  approval  of  the 
Department’s  budget;  continues  through  the 
planning,  soliciting,  evaluating,  awarding,  and 
administering  processes;  and  concludes  with 
final  contract  closeout.  Specifically,  each 
review  includes  the  following: 

•  An  examination  of  acquisition  manage¬ 
ment  practices 

•  An  appraisal  of  the  professionalism  of 
the  acquisition  work  force 

0  An  analysis  of  the  organizational  place¬ 
ment  and  structure  of  the  agency’s 
acquisition  function 

•  An  assessment  of  the  responsiveness  of 
the  acquisition  work  force  to  the  needs  of 
its  customers 

•  An  evaluation  of  the  extent  of  com¬ 
pliance  with  laws,  regulations,  and 
policies 

•  An  evaluation  of  acquisition  process 
flow,  including  automation  and  inno¬ 
vative  technology  applications 

•  An  assessment  of  the  adequacy  of  the 
acquisition  management  information 
system 

•  An  appraisal  of  the  acquisition  organi¬ 
zation’s  structural  checks  and  balances. 

When  engaged  by  agencies  to  evaluate 
their  acquisition  management  practices  and 
recommend  improvements,  LMI  was 
challenged  to  develop  a  review  methodology 
that,  although  tailored  to  each  agency’s  unique 
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requirements  and  practices,  nevertheless  pro¬ 
vided  consistent  standards  for  measuring  the 
quality  of  the  acquisition  process  and  for  rating 
each  organization.  The  methodology  enables 
designated  teams  to  conduct  reviews  efficiently 
and  cost-effectively,  with  minimum  disruption 
to  the  agency  under  review,  while  at  the  same 
time  ensuring  that  the  review  produces 
defensible  results. 

As  we  look  at  the  involvement  of 
contracting  personnel,  project  officers,  and 
management  in  the  acquisition  process,  we 
place  primary  emphasis  on  day-to-day 
acquisition  operations.  Using  this  approach, 
we  have  helped  offices  .streamline  their  acqui¬ 
sition  processes  to  improve  quality,  timeliness, 
productivity,  and  responsiveness  to  customers. 

This  article  describes  our  procedures  for 
performing  comprehensive  acquisition  ma¬ 
nagement  reviews.  In  the  history  section,  we 
highlight  the  Government-wide  requirements 
mandating  such  reviews.  Then  we  briefly 
discuss  some  of  the  methods  used  by  others  to 
conduct  various  types  of  acquisition  reviews. 
Finally,  we  describe  the  review  methods  and 
strategies  we  developed  while  conducting 
acquisition  reviews  at  nine  different  civil 
agencies. 

Some  of  the  innovative  methods  and 
applications  described  include  the  use  of 

•  Management  indicators  and  contracting 
benchmarks 

•  Process  flow  analysis  techniques  as  a 
means  for  assessing  the  effectiveness  of 
both  program  offices  and  contracting 
offices  in  carrying  out  the  process 

•  Interviews  for  primary  data  collection 
and  to  corroborate  and  expand  on  infor¬ 
mation  generated  from  all  primary 
sources 

•  Methods  for  identifying  the  appropriate 
size  and  stratification  of  the  sample  on 
the  basis  of  the  contract  file  universe’s 
size  and  composition,  and  for  selecting 
which  specific  files  will  be  reviewed. 

•  Questionnaire  review  checksheets 
geared  to  various  types  of  contracts  and 


addressing  eight  different  aspects  of  the 
acquisition  process  (including  a  scoring 
method  for  each) 

•  An  overall  scoring  methodology  for  nu¬ 
merically  rating  the  quality  of  the 
agency’s  acquisition  operations 

•  Alternative  solutions  for  correcting  defi¬ 
ciencies 

HISTORY 

Executive  Order  12352,  “Federal 
Procurement  Reforms,"  of  March  17,  1982 
requires  that  the  head  of  each  executive 
agency 

.  .  .  designate  a  Procurement  Executive 
witli  agency-wide  responsibility  to  oversee 
development  of  procurement  .system.s,  eva¬ 
luate  system  performance  in  accordance 
with  approved  criteria,  enhance  career 
management  of  the  procurement  work 
force,  and  certify  to  the  agency  head  that 
procurement  systems  meet  approved 
criteria  .... 

In  order  to  aid  agency  procurement 
executives  in  fulfilling  their  certification  res¬ 
ponsibilities  required  by  the  Executive  Order, 
the  Office  of  Federal  Procurement  Policy 
<OFPP)  formed  a  multi-agency  team.  Its 
mis.sion  was  to  develop  procurement  system 
certification  criteria  as  guidance  in  evaluating 
procurement  systems.  Recognizing  the  consi¬ 
derable  diversity  among  the  agencies,  the  team 
developed  a  comprehensive  list  of  criteria 
covering  most  situations,  thereby  enabling 
each  agency  to  select  the  criteria  best  meeting 
its  unique  needs.  It  was  the  team’s  philosophy 
that  no  uniform  Government-wide  set  of  re¬ 
view  standards  should  be  required.  As  a 
result,  each  agency  was  authorized  to  apply  its 
owm  review  methodology  when  certifying  the 
acceptability  of  its  procurement  system  to  its 
agency  head.  While  we  agree  that  no  one  set  of 
standards  can  apply  across  the  board,  without 
some  uniformity  it  is  difficult  to  measure  an 
acquisition  office's  progress  over  time  or  to 
compare  its  performance  with  that  of  other 
equivalent  acquisition  organizations.  On  the 
other  hand,  substantial  judgment  is  required 
in  the  application  of  any  established  standard. 
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Most  available  writings  on  reviews  of 
procurement  in  the  Government  sector  revolve 
around  studies  of  major  systems  acquisitions 
that  have  sought  to  reform  DoD’s  weapon 
systems  acquisition  process.  Relatively  few 
reports  concern  results  of  reviews  of  individual 
procurement  office  operations.  However,  there 
is  a  larger  body  of  relevant  literature  that 
applies  to  the  commercial  sector;  we  surveyed 
it,  since  the  similarities  between  the 
Government  and  commercial  sectors  have  been 
found  to  be  striking  (1|.  Later  research  found 
that  the  tasks  of  the  purchasing  manager  were 
much  the  same  in  both  sectors,  although  their 
importance  differed  ( 2 1. 

Early  literature  on  procurement,  from 
the  late  1800s,  focused  on  describing  the 
functions  required  to  be  performed  and  the 
requisite  personal  attributes  of  the  individual 
who  would  perform  them  [3|.  In  1964,  the 
American  Management  Association  published 
Eiahiating  Purchasing  Performance,  which 
remains  useful  in  describing  the  issues  to  be 
addressed  during  a  review  (4|.  This  work  had 
been  preceded  by  The  Internal  Audit  and 
Control  of  a  Purchasing  Department  15\.  Since 
these  seminal  efforts,  a  number  of  studies  on 
methodologies  for  evaluating  the  procurement 
function  have  been  published. 

Ammer  found  that  purchasing  depart¬ 
ments  were  not  being  held  to  the  same 
standards  as  other  corporate  departments  161. 
Ammer  believes  that  purchasing,  like  other 
departments,  should  be  measured 
quantitatively  in  relation  to  corporate 
objectives.  Other  authors  have  proposed 
different  sets  of  quantitative  measures  [7,  8,  9|. 
Racdels,  however,  cautioned  that  some  quan¬ 
titative  measures,  such  as  cost  per  order,  may 
not  be  valid,  because  efforts  to  reduce  mate¬ 
rials  costs  may  increase  order-processing  costs 
1 10|.  Since  he  found  the  cost  of  materials  to  be 
the  dominant  factor  in  determining  the  produc¬ 
tivity  of  the  materials  management  function, 
he  looked  to  a  broader  measure  of  total 
productivity  that  would  relate  the  total  output 
of  goods  and  services  to  the  total  input  of 
resources. 

Others  have  written  on  the  broad  issue 
of  which  types  of  measurements  should  be 
used.  These  have  included  efficiency  versus 


proficiency  or  effectiveness  fll,  12,  13).  Van 
Weele  concluded  that  the  measurement  focus 
will  differ,  depending  on  how  management 
looks  at  purchasing.  The  focus  will  be  on 
efficiency  if  purchasing  is  seen  as  a  clerical  or 
commercial  activity,  while  it  will  be  on 
effectiveness  if  purchasing  is  viewed  as  a 
strategic  function  of  the  corporation. 

The  question  of  whether  purchasing 
evaluations  should  be  made  by  an  outsider 
may  have  been  resolved.  Several  authors 
recommend  that  purchasing  audits  encom¬ 
passing  the  entire  range  of  procurement  pro¬ 
cess  activities  should  be  conducted  by  someone 
impartial  [4,  13,  14).  Some  authors  have  also 
concluded  that  comparison  with  other  pur¬ 
chasing  operations  is  not  the  answer  —  that 
“the  best  standard  for  a  purchasing  depart¬ 
ment  is  its  own  past  performance”  (original 
emphasis)  [151.  We  believe  that  the  answer 
probably  depends  on  the  review’s  purpose,  with 
the  inside  review  being  useful  primarily  to  the 
head  of  the  purchasing  office  and  the  outside 
review  being  useful  to  managers  at  a  higher 
level,  who  want  to  be  sure  that  no  conflict  of 
interest  affects  the  results.  We  also  believe 
that  the  outside  review  is  most  beneficial  and 
that  comparisons  can  be  made,  using  judgment 
to  adjust  for  varying  situations. 

On  the  basis  of  this  prior  work  and  of 
unpublished  individual  agency  review  guides 
with  which  we  were  familiar  le.g.,  those  of 
NASA,  GSA,  DOE,  and  DoDi,  we  started  per¬ 
forming  comprehensive  management  reviews. 
Afler  each  review,  we  incorporated  into  our 
evolving  methodology  additional  successful 
techniques  used  or  learned  during  that  review. 
While  we  continue  to  search  for  more  improve¬ 
ments,  we  consider  the  techniques  discussed 
here  to  constitute  an  established,  mature 
approach  to  performing  comprehensive 
management  reviews  of  acquisition  operations 
and  systems. 

REVIEW  METHOD 

As  analysts,  our  primary  aim  is  to 
improve  acquisition  management  processes. 
Our  reviews  examine  acquisition  management 
practices  as  implemented  by  contracting 
officers,  project  officers,  and  others  involved  in 
the  proces.s.  In  our  reviews,  we  emphasize  a 
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comprehensive,  bottom-up  operational 
approach  in  which  we  examine  the  various 
steps,  organizations,  and  activities  required  for 
acquisition,  starting  with  the  determination  of 
a  need  and  passing  through  work-statement 
preparation,  solicitation,  evaluation,  negotia¬ 
tion,  contract  award,  and  administration  to 
contract  closeout.  We  examine  all  aspects  of 
the  acquisition  cycle,  spending  as  much  effort 
in  reviewing  the  program  and  project  offices  as 
in  reviewing  the  contracting  offices.  In  this 
way,  we  believe  we  are  best  able  to  provide  a 
balanced  point  of  view,  to  identify  areas  that 
need  improvement,  and  to  make  specific 
recommendations  on  fixing  the  system.  Our 
underlying  principle  is  that  management,  in 
order  to  be  able  to  judge  the  quality  and 
effectiveness  of  its  acquisition  activities,  must 
have  (1)  accurate  knowledge  of  the  workings  of 
the  acquisition  process  and  (2)  a  means  of 
measuring  the  contribution  of  each  organi¬ 
zation  involved  in  acquisition.  Figure  1  pro¬ 
vides  an  overview  of  the  comprehensive  acqui¬ 
sition  review  process. 

The  acquisition  cycle  begins  once  it  has 
been  determined  that  a  product  or  service  is 
needed  and  it  ends  when  the  product  or  service 
has  been  delivered  and  the  contract  is  closed 
out.  We  review  the  entire  decision-making 
process  associated  with  the  acquisition  cycle, 
examining  both  its  program  and  contracting 
features.  This  approach  allows  us  to  uncover 
the  causes  of  deficiencies  and  to  identify  areas 
requiring  improvement.  With  this  informa¬ 
tion,  we  can  then  provide  specific  recommenda¬ 
tions  for  corrective  actions. 

Preparation  for  Review 

Before  starting  on-site  reviews,  we  lay 
the  groundwork  by  requesting  that  our  sponsor 
arrange  a  meeting  for  us  with  the  management 
representative  of  the  office  to  be  reviewed. 
Our  objective  is  to  have  our  sponsor  introduce 
LMI  and  to  describe  the  review’s  objectives  and 
the  need  for  cooperation.  Another  objective  of 
the  meeting  is  to  address  any  concerns 
management  may  have  about  the  review 
process. 

The  review  process  actually  begins  in 
our  own  offices,  well  in  advance  of  the  start  of 
the  on-site  review  phase.  We  plan  our  review 


strategy  and  prepare  a  request  for  information 
regarding  the  agency’s  acquisition  activities. 
The  requested  data  include  preliminary 
information  about  the  agency’s  acquisition 
organization,  its  staffing,  and  its  management 
practices  and  a  listing  of  contracts  awarded 
during  the  past  2  fiscal  years.  Some  of  the  data 
requested  are  quite  specific,  such  as  the 
number  and  types  of  awards  obligated  each 
month;  socioeconomic  goals  versus  actual  expe¬ 
rience;  number  of  actions  completed  but  not 
closed  out;  a  roster  of  key  management, 
program,  acquisition  management,  and  other 
personnel  involved  in  the  award  and  moni¬ 
toring  process  with  the  organizations  to  which 
they  belong;  the  organizational  placement  and 
structure  of  the  acquisition  organization;  and 
office  productivity  and  timeliness.  Other 
information  requested  is  more  general  in 
nature  and  relates  to  such  areas  as  manage¬ 
ment  and  personnel  practices,  regulations  and 
policies,  and  oversight  responsibilities. 

Organizational  Profile 

After  we  receive  preliminary  data,  we 
begin  to  develop  a  profile  of  the  agency’s 
acquisition  activities. 

We  use  management  indicators  and 
contracting  benchmark’s  predetermined  stan¬ 
dards  to  conduct  our  analysis.  Management 
indicators  describe  the  ability  to  manage  the 
contracting  function  and  the  people  in  it, 
rather  than  the  ability  to  operate  the  hands-on 
contracting  process  itself.  Examples  include 
span  of  control  (number  of  personnel  reporting 
to  a  single  supervisor)  and  the  extent  of 
employee  turnover  and  overtime.  Contracting 
benchmarks,  on  the  other  hand,  address  direct 
contracting  issues.  Examples  include  degree 
of  competition,  the  number  of  delinquent 
deliverables,  and  the  extent  of  cost  growth.  By 
comparing  the  reported  data  with  the 
predetermined  standards,  we  can  quickly 
determine  whether  a  given  activity  falls 
within  an  acceptable  range  of  performance.  If 
not,  the  review  team  flags  that  area  as  one  to 
be  specifically  followed  up  on  during  the  on¬ 
site  review  phase.  From  the  roster  of  per¬ 
sonnel,  we  prepare  a  preliminary  list  of  people 
to  be  interviewed.  The  list  is  meant  to  in¬ 
clude  a  broad  cross-section  of  the  individuals 
and  organizations  involved  in  the  acquisition 
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FIG.  1.  COMPREHENSIVE  ACQUISITION  REVIEW  PROCESS 
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process  -  from  upper  management  to  suppor¬ 
ting  personnel. 

The  On-Site  Review  -  Entrance  Interview 

Once  the  preliminary  data  have  been 
assembled  and  analyzed,  we  start  the  actual 
on-site  review  process  at  the  contracting  office. 
First,  we  arrange  an  entrance  interview  with 
the  head  of  the  office  being  reviewed,  or  a 
designee.  During  this  interview,  we  describe 
the  purpose  of  our  review  and  our  objectives, 
methods,  and  strategies.  We  ask  that  the 
interviewees  identify  any  special  concerns  they 
may  have  for  the  team  to  review.  We  request 
that  they  prepare  a  letter  to  the  staff  intro¬ 
ducing  the  review  team  and  that  they  appoint 
a  liaison  individual  to  help  obtain  data  and 
clear  the  way  for  the  interviews. 

Review  Teams 

We  usually  conduct  comprehensive 
reviews  using  two-person  teams.  The  number 
of  teams  depends  on  the  size  and  complexity  of 
the  organization  under  review.  Each  team  is 
assigned  a  different  aspect  of  the  review:  one 
team  follows  the  process  work  flow,  another 
reviews  contract  files,  and  a  third  interviews 
program'  project  officers.  The  teams  work  in 
parallel  rather  than  sequentially.  The  teams 
meet  frequently  to  discuss  their  ob.servations 
from  the  preceding  day  and  brainstorm, 
building  a  synergy  among  themselves.  They 
share  information  and  use  information 
furnished  by  the  other  teams  to  initiate  new 
areas  of  inquiry.  The  teams  confirm  the 
results  of  each  other’s  investigations.  Occasio¬ 
nally  we  conduct  a  limited  review  restricted  to 
examining  contract  files  and  conducting  some 
incidental  interviews. 

The  Process  Flow  Team 

The  process  flow  team  begins  its  overall 
review  by  summarizing  the  workload  by 
contract  type.  Specific  contract  types  are  then 
traced  through  each  step  in  the  acquisition 
process  from  budget  approval  through  award, 
monitoring,  and  final  closeout.  In  our  reviews, 
we  assume  that  the  acquisition  cycle  begins  at 
the  point  where  it  is  determined  that  a  product 
or  service  is  needed.  The  team  begins  its 


analysis  by  identifying  the  individuals  and 
organizations  involved  in  each  step  of  the 
process.  Next,  the  team  measures  il)the 
actual  hands-on  working  hours  expended  for 
each  process  step,  and  i2)  the  number  of  days 
elapsed  from  its  beginning  to  its  completion.  It 
compares  that  number  with  the  agency’s  time 
standard  for  completing  the  step.  The  team 
then  determines  why  a  specific  individual  or 
organization  is  involved  in  one  or  more  steps  of 
the  process,  by  identifying  the  law.  regulation, 
policy,  or  practice  requiring  that  involvement. 
Finally,  the  team  attempts  to  determine  the 
extent  to  which  that  involvement  adds  value, 
and  the  additional  time  required  by  that 
involvement. 

In  most  agencies  reviewed,  we  found 
that  neither  the  process  flow  step.>^.  nor  the 
related  processing  times  for  each  .step  had  been 
documented.  As  a  consequence,  it  was  neces¬ 
sary  for  us  to  document  the  actual  steps,  and 
the  processing  time  associated  with  each. 
Figure  2  is  an  example  of  a  small  segment  of  a 
typical  contract  process  flow  diagram.  Each 
step  is  identified  by  a  separate  box.  It  contains 
the  task  number,  the  start  date,  its  duration  in 
days,  a  brief  description  of  the  activity,  the 
responsible  individual  or  organization,  the 
actual  hands-on  working  hours  expended,  and 
the  law.  regulation,  policy,  or  practice  that 
requires  it. 

The  Contract  File  Review  Team 

From  the  list  of  contracts  provided 
covering  the  preceding  2  fiscal  years,  we  .select 
a  random  or  stratified  random  sample  of 
contracts  to  be  reviewed  during  our  on-site 
activities.  To  determine  the  sample  size,  the 
review  team  considers  several  factors.  The 
team’s  goal  is  to  select  a  sample  that  i.s  no 
larger  than  that  required  to  achieve  the 
accuracy  that  is  needed.  It  is  recognized  that, 
generally,  the  larger  the  sample,  the  more 
accurately  it  will  reflect  the  population;  but  it 
will  also  take  longer  to  review  and  analyze. 
The  team  normally  selects  a  sample  size  that 
will  give  them  99  percent  confidence  that  the 
average  sample  scores  will  be  within  plus  or 
minus  .15  of  the  population  average.  For 
example,  if  the  team  scores  the  contract  files  at 
2.45,  they  can  be  confident  in  saying  that  the 
average  score  of  all  files  is  between  2.30  and 
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FIG.  2.  PORTION  OF  A  TYPICAL  CONTRACT  PROCESS  FLOW  DIAGRAM 

(Hypothetical) 


2.60  (2.45  plus  or  minus  .15).  Table  1  shows 
the  necessary  sample  size  to  be  selected  for  a 
given  population  size. 

TABLE  1 

DETERMINING  SAMPLE  SIZE 


Population  size 

Sample  size 

5 

4 

10 

6 

15 

8 

20 

9 

30 

10 

50 

12 

75 

13 

100 

13 

200 

14 

300 -F 

15 

Note:  Sample  standard  deviation  assumed  to 
be  no  greater  than  0.2.  See  Clelland,  Richard  C., 

John  S.  deCani,  and  Francis  E.  Brown,  flasrc 
Statistics  with  Business  Applications,  (New  York: 

John  Wiley  and  Sons,  Inc  ),  1973,  p  323. 

The  team  responsible  for  reviewing  the 
contract  files  evaluates  and  scores  the  follo¬ 
wing  eight  functional  areas  for  each  contract 
reviewed,  using  a  contract  file  review 
checklist: 

•  Advance  acquisition  planning 

•  Purchase  request 

•  Solicitation 

•  Proposal  evaluation 

•  Discussions/negotiations 

•  Contract  award 

•  Management  reviews 

•  Contract  administration. 


These  eight  functional  area  scores  are 
then  summarized,  and  an  overall  score  for  the 
individual  contract  file  is  computed.  Exhibit  1 
is  a  contract  file  review  checklist  used  for 
advance  acquisition  planning  'for  negotiated 
contracts),  one  of  the  eight  functional  areas. 
The  other  seven  areas  are  reflected  in  similar 
checklists,  each  specifically  designed  for  the 
function  being  reviewed.  Each  checklist  has 
two  parts.  The  first  part  contains  questions 
that  must  be  answered  by  the  team  during  the 
review.  The  second  part  sets  forth  a  rating 
scheme  to  help  the  reviewers  evaluate  that 
functional  area. 

Once  the  review  of  the  contract  files  are 
complete,  the  team  prepares  a  scoring  matrix 
in  which  the  eight  functional  area  scores  and 
the  overall  score  for  each  contract  file  are 
posted.  Next,  an  overall  average  score  is 
computed  for  each  functional  area  and  for  the 
contracts  taken  as  a  whole.  Each  functional 
area’s  overall  score  is  then  compared  with  the 
predetermined  standard  score  of  2.5  to  deter¬ 
mine  the  extent  to  which  the  acquisition 
activity  meets  that  standard.  For  the  activity 
to  be  considered  totally  acceptable  under  our 
scoring  approach,  each  separate  functional 
area  must  also  meet  the  standard.  Thus,  even 
if  the  overall  score  for  all  contract  files 
reviewed  meets  or  exceeds  the  standard,  but 
one  or  more  functional  area  is  below  standard, 
a  full  explanation  of  the  deficiency  is  presented 
in  our  report  of  findings.  We  feel  it  imperative 
to  bring  the  deficiency  to  management’s 
attention  so  that  corrective  action  can  be 
taken.  Since  we  use  a  statistically  random 
sampling  method  in  selecting  files  for  review,  a 
deficiency  noted  in  our  sample  is  likely  to  be 
repeated  in  all  contract  files.  Table  2  is  a 
matrix  that  summarizes  the  contract  file 
review  scoring  results  by  individual  contract 
files  (identified  only  by  letter  and  not  by  name, 
to  preserve  confidentiality)  and  by  functional 
area,  computing  an  overall  average  score  for 
each.  The  summary  information  from  this 
table  is  then  graphically  displayed  as  in 
Figures.  While  the  contract  file  review  may 
provide  only  one  measurement  of  the  quality  of 
the  work  performed  by  the  acquisition  organi¬ 
zation,  we  weigh  it  most  heavily  in  our  scoring 
formula  because  the  official  files  are  required 
to  contain  the  full  history  of  all  of  the  activities 
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EXHiBiTI 


CONTRACT  FILE  REVIEW  CHECKLIST 


ADVANCE  ACQUISITION  PLANNING  (NEGOTIATED  CONTRACTS) 

IF  NO  PLAN  DONE  AND  NONE  REQUIRED,  GO  ON  TO  NEXT  PAGE 

YES 

NO 

ACQUISITION  PLAN  REQUIRED: 

□ 

□ 

ACQUISITION  PLAN  PREPARED: 

□ 

□ 

CONTRACTING  OFFICE  INVOLVED  IN  ACQUISITION  PLANNING: 

□ 

□ 

MILESTONE  SCHEDULE  IN  FILE: 

□ 

□ 

ACQUISITION  PLAN  CONTENT  MEETS  REQUIREMENTS: 

□ 

□ 

PLAN  SIGNED  BY  PROJECT  OFFICER:  DATE 

□ 

□ 

PLAN  SIGNED  BY  CONTRACT  SPECIALIST:  DATE 

□ 

□ 

CONTRACTING  OFFICER  APPROVED  ($100,000  TO  $1,000,000): 

□ 

□ 

MANAGEMENTAPPROVED  (OVER  $1,000,000): 

□ 

□ 

CONTENT  OF  PLAN  AND  QUALITY  OF  ADVANCE  ACQUISITION  PLANNING  RATING 

RATING 

REQUIRED  PLAN  NOT  DONE  OR  SERIOUSLY  INADEQUATE: 

0  TO  1 .49 

INADEQUATE  PLAN  QR  PREPARED  WITHQUT  CONTRACT  SPECIALIST 
PARTICIPATION: 

1.50  TO  2.49 

PLAN  MINIMALLY  MEETS  REGULATORY  REQUIREMENTS: 

2.50 

PLAN  EXCEEDS  MINIMUM  REGULATORY  REQUIREMENTS,  CONTAINS 

SIGNIFICANT  PROCUREMENT  STRATEGY,  AND  INVOLVES  CONTRACTING  OFFICE 
EARLY  IN  THE  PROCESS: 

2.51  TO  3.49 

PLAN  SO  SUPERIOR  THAT  IT  CLEARLY  CONTRIBUTES  TO  MISSION 
ACCOMPLISHMENT,  CONSIDERING  COST,  QUALITY,  AND  SCHEDULE: 

3.50  TO  4.0 

RATING  ASSIGNED 
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TABLE  2 


SUMMARY  OF  CONTRACT  FILE  REVIEW  RESULTS 
(Hypothetical) 


File 

Ad¬ 

vance 

acquisi¬ 

tion 

plan¬ 

ning 

Purchase 

request 

Solicita¬ 

tion 

Proposal 

evalua¬ 

tion 

Discus¬ 

sions/ 

negotia¬ 

tions 

Con¬ 

tract 

award 

Manage¬ 

ment 

reviews 

Con¬ 

tract 

istra- 

tion 

Total 

Avg. 

a 

2.6 

3.0 

3  2 

2.8 

2.6 

3  0 

3.1 

2.0 

22  3 

2  8 

b 

1.5 

2.1 

2.4 

18 

2.0 

2.2 

2.1 

1.5 

- 1 

1d.6 

20 

c 

n/a 

2.4 

2.6 

2.1 

2.3 

2.3 

2  4 

18 

15.9 

2  3 

y 

- 1 

Z 

Total 

28.5 

38.0 

41.6 

33  5 

33  0 

37  0 

37  5 

26  1 

275.2 

Avg. 

2.0 

2.5 

2.8 

2.2 

2.2 

2.: 

2.5 

1.7 

Note:  Ratmq:  3  50  to  4.00:  iubstantially  exceeds  standard;  2  $1  to  3  49.  -'xceeds  standard:  2  50  meets  stanaard,  1  50  to  2  49  beiow  stanaara,  0  to  t  49 
substantially  below  standard 


related  to  awarding  and  administering  the 
contract. 

The  Interview  Team 

A  major  portion  of  the  interview  team’s 
time  is  devoted  to  assessing  the  role  and 
effectiveness  of  both  project  and  contracting 
officers  during  the  contract  award  and 
monitoring  process.  The  project  officer  is  the 
contracting  officer’s  customer,  as  the  person 
with  the  requirement  that  must  be  filled.  The 
project  officer  plays  a  key  role  in  requirements 
definition,  planning,  presolicitation  proces¬ 
sing,  technical  evaluation,  and  contract 
monitoring.  When  the  review  team  evaluates 
the  effectiveness  of  the  project  officer  through 
the  interview  process,  it  uses  the  analysis 
prepared  by  the  process  flow  team,  the  results 
of  the  contract  file  review,  and  the  results  of 
the  evaluation  of  the  management  indicators 
and  contracting  benchmarks.  The  following 
specific  points  are  usually  addressed  by  the 
interview  team; 


•  Whether  the  project  officer’s  stated  role 
is  being  carried  out  effectively 

•  Whether  required  actions  are  taken  in  a 
timely  manner  and  are  thoroughly  and 
effectively  accomplished 

•  Whether  the  project  officer  is  tailoring 
actions  to  meet  differing  individual 
requirements  racher  chan  using  a 
generic  approach 

•  'vVhether  the  project  officer  is  notifying 
the  contracting  officer  of  delinquent  or 
unacceptable  deliverables  in  a  timely 
manner 

•  Whether  the  project  officer  is  providing 
the  contracting  officer  regular  reports  of 
the  contractor's  progress,  including  an 
assessment  of  the  contractor's  progTe>s 
reports. 

The  team  then  interviews  representa¬ 
tives  of  the  contracting  organization,  including 
contract  specialists,  contracting  officers, 
cost'price  analvsts,  and  supervisors  in  that 
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Meets 

standard 


4  00 


3  00 


2.00 


1.00 


FIG.  3.  CONTRACT  FILE  REVIEW  RESULTS 
(Hypothetical) 


organization.  Finally,  it  interviews  others, 
such  as  the  Small  and  Disadvantaged  Business 
Utilization  Specialist  iSADBUS),  property 
administrators,  finance  officers,  quality 
assurance  representatives,  legal  counsel, 
inspector  general  representatives,  and  the 
competition  advocate.  Because  each  has  some 
involvement  in  the  acquisition  process,  each 
has  a  unique  story  to  tell  about  the  process  and 
how  it  can  be  improved. 

The  interviewers  attempt  to  address  the 
functional  areas  being  rated  in  the  contract  file 
review,  the  management  indicators,  and  the 
contracting  benchmark  areas.  The  interviews 
focus  on  two  areas;  the  knowledge,  training, 
and  experience  of  the  interviewee,  and  the 
interviewee’s  specific  involvement  in  the 
acquisition  process.  The  interviewee  is 
advi.sed  that  the  minutes  of  the  meeting  will  be 
used  to  help  highlight  operational  trends  and 
will  remain  part  of  the  team’s  official  work 
papers. 


After  evaluating  each  management 
indicator  and  contracting  benchmark,  tne 
interview  review  team  assigns  each  functional 
area  a  descriptive  rating.  If  the  functional 
area  appears  to  be  well  managed,  with  the 
work  correctly  and  effectively  performed,  then 
that  functional  area  is  usually  rated  as 
meeting  the  standard.  Ratings  above  or  below 
standard  are  assigned  where  strengths 
predominate  or  where  the  need  for 
improvement  merits  serious  consideration. 
Exhibit  2  is  a  hypothetical  completed  manage¬ 
ment  indicators  worksheet,  while  Figure  3  is  a 
completed  hypothetical  contracting  bench¬ 
marks  worksheet.  In  evaluating  each  factor, 
the  team  uses  multiple  sources  of  information. 
The  primary  information  sources  are 

•  Responses  to  the  initial  request  for  data 

•  Responses  to  the  contracting  activity 
profile 

•  Responses  to  questions  posed  during  the 
interview  phase 
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•  Examination  of  supporting  documenta¬ 
tion 

•  Interaction  with  contract  file  review  and 
process  flow  team  members. 

Each  team  member  prepares  a  two-part 
written  justification  to  support  the  rating 
selected.  The  first  part  contains  essential 
background  (i.e.,  a  summary  description  of  the 
characteristics  and  importance  of  the  topic  to 
be  rated,  and  the  requirements  or  factors 
considered  in  the  rating,  including  regulatory 
ones).  The  second  part  normally  includes 
observations  and  discoveries  made  during  the 
review,  discussions  of  any  particularly  notable 
attributes  or  practices  leading  to  ratings  above 
or  below  standard,  and  any  other  explanations 
necessary  to  provide  insight  concerning  the 
assigned  rating.  As  a  final  check  on  the 
ratings  assigned,  at  the  conclusion  of  the 
process  the  responsible  team  member  presents 
an  oral  defense  of  each  rating  to  the  entire 
review  team.  Any  additional  pertinent  infor¬ 
mation  developed  by  members  of  the  contract 
file  review  or  process  flow  teams  is  factored 
into  the  ratings  at  that  time.  Once  the  ratings 
and  supporting  documentation  are  agreed  to  by 
the  entire  team,  the  narrative  descriptions  are 
then  converted  to  numeric  equivalents  for  ease 
of  presentation.  A  numeric  scale  of  0  to  4.0  is 
used,  with  4.0  being  the  highest  score  and  2.5 
considered  the  standard  score.  (See  Figure  3) 

The  information  gathered  during  the 
interview  process  helps  to  confirm  the  results 
of  the  findings  from  the  contract  file  review 
and  process  flow  teams.  However,  there  is  no 
separate  numeric  scoring  assigned  for  program 
or  project  management  activities.  Any  areas 
requiring  improvement  are  addressed  in 
narrative  fashion  in  the  final  report. 

THE  FINAL  REPORT 

Upon  completion  of  the  review,  the 
entire  review  team  develops  a  final  overall 
numeric  rating  for  each  functional  area.  In 
arriving  at  these  numeric  ratings,  the  review 
team  converts  the  narrative  description  to 
numeric  equivalents  for  ease  of  presentation. 
Finally,  the  team  merges  the  management 
indicators,  contracting  benchmarks,  and 


contract  file  review  scores  to  develop  an  overall 
rating  for  the  acquisition  organization. 

We  present  the  office  head  with  an  oral 
briefing  of  our  findings  at  the  conclusion  of  our 
review.  We  indicate  at  that  time  though  the 
organization’s  overall  average  may  meet  or 
exceed  the  standard,  functional  areas  falling 
below  this  standard  will  be  addressed  in  the 
final  report.  Subsequently,  our  sponsor  sends 
the  office  head  a  copy  of  the  final  report  and,  if 
there  are  areas  requiring  improvement,  re¬ 
quests  that  a  corrective  action  plan  be  sub¬ 
mitted  along  with  an  implementation  sche¬ 
dule. 

SUMMARY 

LMI  uses  the  strategies  described  in  this 
article  to  conduct  its  comprehensive  acqui¬ 
sition  management  reviews.  By  using  a 
structured  review  methodology,  we  have  found 
that  teams  conduct  their  reviews  in  a 
consistent  manner  from  organization  to  orga¬ 
nization.  Our  method  helps  agency  internal 
review  teams  in  the  same  way.  This  consis¬ 
tency  ensures  fairness  and  enables  the  identi¬ 
fication  of  common  problems  in  agencies. 
Therefore,  we  are  in  a  unique  position  to 
identify  common  trends  based  on  our  findings. 

We  also  believe  that  our  reviews  get  to 
the  heart  of  an  organization’s  problems 
quickly.  This  becomes  particularly  true  when 
management  uses  the  process  flow  analysis  to 
compare  program  officers’  involvement  and 
contributions  in  the  preaward  and  postaward 
acquisition  phases  with  the  involvement  and 
contributions  of  the  contract  management 
organization.  When  upper  management  is 
provided  a  review  based  on  a  scoring  metho¬ 
dology,  it  has  a  quantified  basis  forjudging  the 
effectiveness  with  which  its  acquisition  opera¬ 
tions  are  being  conducted. 
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ABSTRACT 

Toward  the  end  of  the  fourth  quarter, 
the  contracting  offices  in  many  Federal 
agencies  undertake  frantic  efforts  to  obligate 
their  fiscal-year  funds  before  year-end.  This 
scramble,  which  may  result  in  higher 
processing  cost  and  lower  quality  in  the  award 
process,  often  leads  to  an  erroneous  perception 
by  higher  agency  management  that  the  failure 
to  space  workload  evenly  throughout  the  fiscal 
year  is  attributable  primarily  to  poor  planning 
and/or  execution  by  the  contracting  office  staff. 
Over  the  years,  this  perception  has  led  to  the 
commissioning  of  a  number  of  high-level 
studies  that  have  focused  primarily  on  the 
contract  award  function  rather  than  looking  at 
the  acquisition  process  as  a  whole. 

We  studied  the  overall  acquisition 
systems  at  several  agencies  and  found  the 
primary  cause  of  late  fourth  quarter  contract 
awards  to  be  the  failure  of  program  offices  to 
submit  completed  procurement  requests  to 
contracting  offices  in  a  timely  manner  While 
program  officers  often  excuse  their 
delinquency  by  blaming  Congress  for  failing  to 
appropriate  funds  in  a  timely  fashion,  we 
generally  found  this  argument  to  be  without 
merit,  since  the  planning  process  can  actually 
start  well  before  funds  are  appropriated. 

To  better  understand  the  linkage 
between  planning,  budgeting,  and  project 
implementation,  and  to  develop  a  basis  for 
streamlining  and  simplifying  the  acquisition 
process,  we  applied  a  process  flow  analysis  tool. 
This  tool  provides  for  charting  the  steps  in  the 
process;  identifies  the  law,  regulation,  policy, 
or  practice  requiring  each  .step,  the  start  date 


and  actual  elapsed  time  for  that  step,  and  the 
actual  professional  and  support  hands-on 
hours  expended  for  completing  it.  Visual 
display  is  achieved  by  using  commercial  PC 
software  for  on-screen  and  hardcopy  printout. 

The  process  flow  analysis  tool  enables 
managers  at  all  levels  to  view  the  entire 
spectrum  of  the  acquisition  process,  identify 
impediments  and  excessive  or  unnecessary 
steps,  and  fix  organizational  responsibility. 
This  paper  describes  the  methodology  we  used 
to  develop  the  tool  and  tells  how  we  applied  it 
to  the  acquisition  process  at  a  number  of 
agencies. 

INTRODUCTION 

The  country  will  no  longer  be  doing 
business  as  usual.  After  12  years  of 
Republican  control  of  the  executive  branch,  the 
reins  have  now  been  turned  over  to  a 
Democratic  Administration.  This  event  cou¬ 
pled  with  the  existence  of  a  Democratic  majo¬ 
rity  in  both  houses  of  Congre.ss,  makes  ma.ssive 
change  possible  at  every  level  of  the  Federal 
Government  and  creates  the  opportunity  for 
the  new  Cabinet  Secretaries  and  other  political 
appointees  to  reshape  their  departments,  as 
needed,  to  carry  out  their  programs 

Most  of  the  appointees  are  new  to  the 
departments  they  will  be  responsible  for 
directing  and  are  thus  unfamiliar  with  the 
departments’  inner  workings.  They  will  quick¬ 
ly  discover  that,  in  many  of  those  departments, 
much  of  their  mission  is  carried  out  by  the 
private  sector  through  contracts  (or  grants) 
despite  the  thousands  of  employees  on  the 
payroll.  They  will  also  find  that  much  of  the 
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work  done  by  the  Federal  bureaucracy, 
consists  of  preparing  budgets  to  support  the 
Administration’s  programs,  preparing  the 
various  documents  necessary  to  award  con¬ 
tracts,  and  monitoring  progress  to  assure  that 
the  purpose  of  the  contract  is  actually  fulfilled. 

While  these  may  not  appear  to  be  very 
daunting  tasks,  the  reality  is  that 
accomplishing  the  award  and  monitoring  of  a 
contract  is  a  complex,  time-consuming,  too 
often  frustrating  exercise  that  requires  real 
skill  and  considerable  knowledge  of  the  depart¬ 
ment’s  internal  operations,  the  steps  and 
players  involved,  and  of  the  vast  body  of 
statutes,  regulations,  policies,  and  practices 
governing  the  process.  “Contracting  [out  of 
services!  is  one  of  the  most  difficult  methods  a 
public  organization  can  choose,  because  wri¬ 
ting  and  monitoring  contracts  require  so  much 
skill.  Many  governments  act  as  if  their  job  is 
done  once  they  have  signed  a  contract. ’’[1| 

Any  manager  faced  with  the  task  of 
implementing  programs  through  contracts  has 
a  responsibility  to  ensure  that  the  awarding 
and  monitoring  process  is  handled  in  an 
effective,  efficient,  and  timely  manner  while 
complying  with  the  applicable  rules.  What 
management  tools  can  the  executive  use  that 
will  quickly  provide  a  full  understanding  of 
what  is  going  on,  to  enable  him  or  her  to 
identify  opportunities  for  improving  the 
process?  One  such  tool  is  a  detailed  process 
flow  diagram.  This  paper  discusses  its 
usefulness  for  controlling  that  part  of  the 
overall  acquisition  process  that  begins  with 
budget  formulation  and  continues  through  to 
contract  award.  The  techniques  we  describe 
can  be  applied  to  other  administrative 
procedures  as  well. 

There  is  a  tremendous  interest  at  all 
levels  of  society  in  the  provision  of  “good 
Government."  In  the  hope  of  ensuring 
efficiency  in  the  provision  of  services,  reform 
efforts  at  the  turn  of  the  century  focused  on 
financial  accountability.  The  need  for  effi¬ 
ciency  continues.  However,  the  Logistics 
Management  Institute  (LMI)  has  found  that 
Government  organizations  seldom  document 
their  work  process  flows.  The  result  is  that 
information  vital  to  the  people  who  must  work 
with  or  evaluate  the  organization’s  operations 


is  not  available.  In  particular,  we  have 
reviewed  several  activities’  acquisition  opera¬ 
tions,  none  of  which  has  had  documented 
process  flow  diagrams.  Instead,  the  organi¬ 
zations  have  been  relying  on  their  staffs  to  use 
their  memories  to  handle  the  next  acquisition 
“just  like  they  did  the  last  one.” 

We  recommend  having  process  flow 
documentation  for  several  reasons.  An 
individual  new  to  the  organization  will  be  able 
to  use  it  as  a  guide  to  learning  how  the 
organization  functions.  Managers  can  review 
sequencing  of  steps,  the  desirability  of 
concurrent  as  opposed  to  sequential  steps,  the 
value  added  by  each  step,  and  the  timeliness  of 
the  process.  Process  improvement  teams  will 
be  able  to  use  it  to  identify  targets  for  review 
(current  total  quality  management  and  similar 
initiatives  must  use  the  existing  process  flow 
as  the  baseline  for  continuous  improvement). 

The  documentation  provides  managers 
and  staff  a  mechanism  for  analyzing  and 
implementing  the  acquisition  process  to 
improve  its  quality,  timeliness,  responsive¬ 
ness,  effectiveness,  and  efficiency.  This  paper 
provides  an  introduction  to  the  development 
and  use  of  process  flow  diagrams  and  their 
analysis  in  the  acquisition  environment, 
focusing  on  moderately  sized  contracting 
activities  conducting  routine  types  of  actions. 

HISTORY 

The  systematic  study  of  process 
management  is  usually  included  as  part  of  the 
basic  college  curriculum  in  production,  with  its 
methods  being  applied  to  the  production  of 
various  types  of  industrial  goods.  Typically, 
the  main  distinction  in  process  flow  from 
organization  to  organization  is  whether  the 
production  operation  is  conducted  using  conti¬ 
nuous  process  flow  or  job  order  shop  manu¬ 
facturing  techniques.  Line  balancing  and 
other  shoploading,  scheduling,  and  sequencing 
problems  are  all  commonly  addressed  by  using 
various  process  management  models,  which 
can  also  be  used  for  controlling  the 
administrative  processes. 

Most  administrative  systems  are 
characterized  by  a  mode  of  operations  in  which 
work  flows  in  serial  fashion  from  one  process  to 
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another.  In  describing  this  type  of  processing, 
Starr  notes  that  (1)  flow  shop  facilities  tend  to 
use  special-purpose  equipment,  well  designed 
to  deliver  the  high  volume  of  output  needed 
and  that  (2)  serial  production  encourages 
division  of  labor  efficiencies  in  accord  with 
specialization  (also,  there  are  economies-of- 
scale  advantages). [2]  While  using  special- 
purpose  equipment  may  not  be  applicable  for 
processing  contract  actions,  conceptually  such 
equipment  serves  as  a  useful  model  by 
suggesting  that  contracting  staffs  need  also  to 
be  highly  specialized  to  accomplish  their 
mission. 

Too  often,  when  a  problem  is  identified, 
a  “patch”  is  applied  to  fix  that  individual 
problem.  A  series  of  patch  fixes  can  soon 
produce  a  poorly  operating  process.  A  broader 
approach  is  needed.  Starr  notes  that  one  the 
main  dangers  we  face  is  the  unwarranted 
treatment  of  problems  in  false  isolation. 
“Success  in  the  design  phase  prohibits  fractio¬ 
nation,  segmentation,  or  splintering  the 
system.”[2]  Looking  at  the  entire  process  flow 
avoids  such  problems. 

As  previously  noted,  near  the  turn  of  the 
century,  a  number  of  public  administration 
reform  efforts  were  designed  to  better  enforce 
accountability.  Stemming  from  those  efforts,  a 
variety  of  methods  of  assessing  accountability 
and  providing  information  for  decisionmaking 
have  been  applied  within  the  government. 
These  include  management  by  objectives, 
performance  budgeting,  operations  research, 
compliance  and  performance  auditing,  pro¬ 
gram  evaluation,  cost  accounting,  financial 
analysis,  and  citizen  surveys. [31  A  current 
requirement  is  the  application  of  continuous 
process  improvement  or  other  total  quality 
management  (TQM)  techniques  in  Federal 
Government  activities. 

An  example  of  this  requirement  is  con¬ 
tained  in  Executive  Order  12637,  dated  April 
28,  1988,  Productivity  Improvement  Program 
for  the  Federal  Government.  It  called  upon 
Federal  Agencies  to  improve  productivity  by 
3  percent  annually.  While  formal  Federal  pro¬ 
ductivity  planning  and  reporting  have  been 
discontinued.  Congress  has  mandated 
reporting  requirements  that  cover  many  of  the 
same  matters.[4] 


Within  the  Government,  an  effort  has 
been  made  to  incorporate  the  techniques  of 
process  flow  into  the  provision  of  public 
services.  They  are  often  incorporated  in  TQM 
programs  as  a  part  of  work  process 
improvement  efforts.  “Processes  are  ‘systems’ 
brought  into  existence  to  produce  results  based 
on  requirements  of  the  customer.  The  systems 
nature  of  work  processes  implies  an 
input/output  structure  where  processing 
contains  a  series  of  value  added  steps.  Inputs 
are  generally  received  from  suppliers  and 
outputs  go  to  customers.”[5] 

Process  flow  techniques  have  been 
adapted  for  use  in  business  applications.  For 
instance,  one  of  the  areas  captivating  many 
managers  today  is  the  use  of  business 
reengineering  techniques  leading  to  simpli¬ 
fication,  streamlining,  and  organizational  redi¬ 
rection.  The  starting  point  for  business 
reengineering  is  charting  the  existing  process 
flow,  a  necessity  before  any  reasonable  systems 
analysis  can  begin.  [6] 

Another  major  impetus  for  process  flow 
charting  and  analysis  within  the  Federal 
Government  results  from  the  Chief  Financial 
Officers  (CFO)  Act  of  1990  and  from  its 
implementation  by  the  Office  of  Management 
and  Budget  (0MB). [7]  The  CFO  Act  applies  to 
fourteen  Federal  departments  and  nine  major 
agencies.  Simply  stated,  it  mandates  that  (1) 
agencies  develop  a  methodology  for  measuring 
performance  of  their  operations  by  using 
organizational  efficiency  and  effectiveness 
performance  indicators  and  that  (2)  program 
outcomes  be  incorporated  into  annual  financial 
statements  for  revolving  and  trust  fund 
activities.  These  requirements  are  imple¬ 
mented  through  various  0MB  and  DoD 
references. 

In  response  to  congressional  and  0MB 
direction,  the  DoD  Comptroller  issued  a 
memorandum  on  October  29,  1992,  to  the 
Service  Secretaries  and  agency  heads  entitled 
“Performance  Budgeting.”  It  says  that  the 
principal  purpose  of  performance  measures  is 
to  gauge  progress  against  stated  goals  and 
objectives.  Other  uses  of  performance  mea¬ 
sures,  including  resource  allocation;  manage¬ 
ment;  and  serving  as  a  basis  for  individual  and 
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organizational  assessment,  recognition,  and 
rewards  are  also  set  forth. 

The  Comptroller  identifies  the  three 
major  categories  of  performance  measures  as 
factor  of  production  indicators,  outcome 
indicators,  and  work  process  indicators: 

•  Factor  of  production  performance  indi¬ 
cators  include  input  measures  relating 
to  efficiency.  Efficiency  measures 
include  unit  cost  per  output,  work  mea¬ 
surement  (labor  input),  and  cycle  time. 
Effectiveness  measures  may  include 
quantity,  timeliness,  quality,  or 
customer  satisfaction. 

•  By  using  impact  measures,  outcome 
indicators  try  to  determine  direct  results 
achieved. 

•  “Work  process  indicators  are  measure¬ 
ments  of  the  way  work  gets  done  in 
producing  the  output;  this  includes  cost- 
effectiveness.  Efficiency  Reviews  or 
Management  Ana  lysis  are  basic 
industrial  engineering-based  approa¬ 
ches  which  flow  chart  the  existing  work 
proces.ses  and  subprocesses.  Efficiency 
reviews  achieve  average  savings  of  10  to 
15  percent  while  competition  (A-76,  etc.) 
produces  savings  of  20  to  30  percent. 
These  reviews  usually  make  limited 
changes  in  work  processes."! 8| 

REVIEW  METHODOLOGY 

For  the  purposes  of  this  paper,  we  are 
primarily  concerned  with  work  process 
indicators.  These  are  indicators  of  the  way 
work  is  accomplished  to  produce  the  output, 
given  a  certain  level  of  re.sources,  efficiency, 
and  effectiveness.  The  Comptroller  describes 
process  flow  charting  as  one  management  tool 
used  in  conjunction  with  work  process 
measurements,  as  follows; 

FIdw  Cliarting  is  a  work  process 
evaluation  tool  that  graphically  maps 
the  activities  that  make  up  a  process 
It  illu.'trates  how  different  elements 
and  ta-iks  and  subtasks  fit  together 
It  IS  an  effective  discipline  to  identify 
all  tasks  and  activities  in  a  work 
proce-,-,  and  the  associated  resources 
par' 1  cii lari V  tiriu-i  associated  with 


those  task.s  and  activities.  Flow 
charts  can  be  designed  to  describe  a 
business  process,  a  functional 
process,  and  the  physical  flow  over 
space  and  time,  all  of  which  provide 
insight  about  efficiency  and 
effectiveness  opportunities  Flow 
charting  deserves  separate  mention 
because  it  is  a  basic  analysis  tool  used 
in  management  analysis,  theorv  of 
constraints,  cost-based  activity 
modeling  and  other  work  process 
evaluation  techniques.|8| 

LMI  uses  process  flow  documentation 
and  analysis  as  one  of  its  evaluation  tools  when 
reviewing  acquisition  management  practices. 
This  tool  enables  us  to  look  across  the  entire 
spectrum  of  an  agency’s  acquisition  proce.ss 
and  track  the  interaction  among  the 
budgeting,  planning,  program,  and  awarding 
offices  that  leads  to  specific  contract  actions.  It 
enables  us  to  identify  (1)  reasons  for  processing 
time  delays  or  uneven  workload  at  any  point 
along  the  acquisition  path.  i2)  the  responsible 
organization.  (3)  impediments  and  bottlenecks, 
and  14)  excessive  or  unnecessary  processing 
steps.  Thus  we  can  establish  organizational 
responsibility  for  successes  as  well  as  failures 
of  the  acquisition  system. 

In  our  experience,  process  flow  studies 
are  best  accomplished  by  a  team  of  two  people, 
at  least  one  of  whom  is  knowledgeable  about 
the  process  being  studied.  This  arrangement 
allows  the  team  to  document  a  set  of  generally 
understood  steps  for  the  process,  so  that 
managers,  new  employees,  evaluators,  and 
others  can  rapidly  understand  the  overall 
requirements  of  the  effort  and  the  typical 
output  requirements  at  any  point  in  the 
process.  In  our  reviews,  we  start  by  examining 
budget  formulation  and  its  linkage  to 
acquisition  and  continue  through  planning 
and  program  office  activities  to  conclude  with 
contract  award. 

Several  different  planning  processes 
take  place  'e.g.,  program  planning  in  support 
of  budget  formulation,  project  planning  in 
.support  of  the  funding  allocation  process,  and 
acquisition  planning,  leading  to  the  award  of  a 
specific  contract).  For  the  purposes  of  this 
paper,  we  focus  on  the  acquisition  planning 
process,  describing  the  means  by  which  the 
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project  and  contracting  offices  work  together  to 
process  a  contract  for  a  specific  project. 

In  our  reviews,  the  acquisiti'  cycle 
begins  with  the  generation  (by  program  office 
personnel)  of  inputs  to  the  annual  advance 
acquisition  plan,  which  is  intended  to 
encompass  all  the  projects  of  the  office.  The 
cycle  progresses  through  various  planning, 
program,  and  contract  office  tasks  until  final 
closeout.  In  the  course  of  our  reviews,  we  have 
not  seen  any  evidence  of  detailed  project 
acquisition  documentation  being  provided  to 
support  the  budget  formulation  process. 
Indeed,  detailed  project  acquisition 
documentation  generally  begins  to  appear  only 
after  the  congressional  budget  implementation 
process  is  well  under  way,  approximately 
1 3  months  after  the  start  of  the  budget  cycle. 

Figure  I  presents  the  typical  steps  taken 
by  LMI  leading  to  the  documentation  and 
analysis  of  the  process  flow.  A  description  of 
the.se  steps  follows. 

The  process  flow  team  begins  its  review 
by  conducting  preliminary  research  on  the 
requesting  component’s  organizational 


struct!'  budgets,  program  types.  Inspector 
Generai/General  Accounting  Office  findings 
and  recommendations,  and  any  other 
significant  published  material.  This  research 
enables  the  team  to  selectively  package  process 
flow  documentation  from  its  prior  studies 
bearing  the  greatest  overall  relevance  to  the 
new  component,  as  an  initial  baseline. 

Initial  presentations  and  discussions  are 
conducted  with  personnel  at  the  organizational 
levels  deemed  appropriate  by  the 
requestor  -  departmental,  divisional,  agency 
and,  ultimately,  bureau  and/or  branch  -  to 
establish  ground  rules  and  goals  for  the 
review.  We  emphasize  at  this  stage  that  the 
end  products  of  the  review  will  be  composites  of 
the  data  provided  by  the  individual  staff 
members  participating  in  the  review,  and  that 
the  process  flow  team  will  maintain  the 
confidentiality  of  personal  responses. 

At  subsequent  working  group  meetings, 
the  specific  contract  actions  that  will  be  the 
subject  of  the  review  are  identified,  as  are  the 
organizational  personnel  who  will  participate. 
These  personnel  gather  available  process  flow 
or  other  documentation  describing  their 
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procedures  for  the  budgeting,  planning, 
programming,  and  contract  award  phases  for 
study  by  the  process  flow  team.  From  these 
documents,  the  process  flow  team  generates 
first  drafts  of  the  suggested  process  flow  steps 
and  data  collection  forms  and  submits  them  to 
the  organizational  personnel  for  final  draft 
edit. 

The  process  flow  team  then  summarizes 
the  organization’s  contracting  workload  by 
contract  purpose  (e.g.,  services,  major  systems, 
evaluations)  and  by  type  (e.g.,  fixed-price,  cost- 
plus-fixed-fee,  or  incentive).  Only  those 
contract  types  having  a  large  number  of 
actions  and  high  dollar  volume  are  usually 
selected  as  our  representative  sample. 
Contracting  types  having  only  a  relatively  few 
examples  are  generally  avoided,  since  the 
sample  size  would  be  too  small  to  be 
representative.  (For  example,  contract  actions 
that  might  be  included  in  a  process  flow 
analysis  could  be  competitive  solicitations  for 
negotiated  services  with  a  price  between 
$100,000  and  $500,000  that  result  in  cost-plus- 
fixed-fee,  single-award  contracts.) 

In  the  various  agencies  that  we 
reviewed,  we  found  that  no  documented 
acquisition  process  flow  diagram  existed  prior 
to  our  efforts  to  develop  one.  In  some  offices, 
process  flow  steps  unique  to  that  individual 
office  had  been  partially  documented  or  could 
be  determined  from  intra-office  memoranda. 
However  most  often  the  only  record  of  the 
process  flow  steps  was  in  the  memory  of  the 
participants,  particularly  for  steps  at  the 
interface  between  components.  In  no  case  was 
the  elapsed  time  for  completing  each  step 
identified,  nor  had  the  average  level  of  applied 
professional  effort  been  determined. 

Because  of  this  recurring  situation,  we 
generally  begin  by  briefly  describing  to  the 
participants  the  process  flow  steps  that  we 
have  previously  developed  at  other  agencies 
considered  to  be  functionally  similar  to  the 
agency  being  reviewed.  The  candidate  process 
flow  diagram  lists  the  description  of  each  step 
considered  relevant  to  the  agency  being 
reviewed.  In  this  initial  discussion,  no  refe¬ 
rence  is  made  to  specific  contract  actions  or 
their  timing;  rather,  the  focus  is  on  the  flow 


itself  and  on  developing  meaningful 
descriptions  for  each  step. 

After  we  have  developed  what  we 
consider  to  be  an  accurate  listing  of  processing 
steps,  we  prepare  a  data  collection  form  to  help 
in  obtaining  basic  information  regarding  each 
step.  Figure  2,  shown  on  the  following  page,  is 
an  example  of  this  form.  The  form  has  a  place 
for  recording  information  about  the  sample 
contract  selected  and  for  describing  its  basic 
characteristics.  The  steps  -  or  milestone 
events  -  are  listed.  The  participants  are 
requested  to  complete  the  form  after  LMI  has 
filled  in  some  initial  data.  For  each  contract  in 
the  sample,  we  require  such  information  as  the 
start  date  and  end  date  for  each  processing 
step;  the  actual  hands-on  processing  time  for 
that  step;  and  the  reference  to  the  law, 
regulation,  policy,  or  practice  requiring  the 
step.  (“Practice"  usually  applies  when  nobody 
seems  to  know  where  the  requirement  came 
from,  other  than  “we’ve  always  done  it  that 
way.")  The  hands-on  processing  time  includes 
the  professional,  clerical,  and  contractor 
support  hours  actually  expended  to  complete 
that  step. 

The  primary  standard  process  flow 
symbol  that  we  use  is  the  activity  indicator, 
represented  by  a  rectangle  with  a  brief 
description  of  the  activity  enclosed.  Figure  3 
contains  an  enlargement  of  the  activity 
indicator  descriptive  legend. 


Task  no. 

Start  date 

Duration: 
work  days 

Task  description 

Task  hours. 

Law,  regulation, 

prof'l/support/contractor 

policy,  practice 

FIG.  3.  EXPANDED  ACTIVITY  BLOCK 

The  example  in  Figure  3  when  filled  in 
includes  such  information  as  the  start  date,  the 
step’s  duration  in  work  days,  hands-on 
processing  time,  and  the  reference  mandating 
the  step.  In  addition,  it  includes  an  alpha¬ 
numeric  descriptor  for  ease  in  identifying  a 
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CONTRACT  DESCRIPTION 
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FIG.  2.  DATA  COLLECTION  FORM  (PARTIAL) 


specific  step.  Since  the  standard  process  flow 
diagraming  symbols  do  not  lend  themselves  to 
depicting  acquisition  process  flow  data,  we 
normally  use  only  the  expanded  activity 
indicator. 

While  some  of  the  required  information 
may  be  available  from  contract  files,  consi¬ 
derable  time  is  usually  expended  during  the 
interviews  in  collecting  data  often  retained 
onl',  in  the  memories  of  the  participants.  We 
cn.phasize  to  the  participants  that  it  is  to  be 
^"xpecte'^  that  the  elapsed  “duration"  time  will 
exr:eed  the  actual  hands-on  processing  time 
(“task  hour<;"i  because  there  will  be  some 
“wait”  time.  On  the  other  hand,  excessive 
elapsed  time  may  be  an  indicator  of  omitted 
steps  and  may  signal  a  need  for  inserting  addi¬ 
tional  steps  in  the  process  flow. 

After  collecting  data  sheets  for  each 
contract  in  the  sample,  we  summarize  the 
individual  actions  into  a  representative  time- 
dependent  process  flow  diagram  for  that  type  of 
contract  action.  Figure  4  is  a  hypothetical 
example  of  a  portion  of  a  typical  contract 
process  flow  diagram.  For  some  events,  actions 
occur  simultaneously  for  all  contracts  (e.g., 
issuance  of  the  planning  document.)  However, 
beginning  with  the  preparation  of  the  purchase 
request  (PR)  and  its  submission  to  the 
contracting  office,  each  purchase  request  is 
treated  as  a  separate  contract  action,  and  the 
date  of  receipt  of  each  in  the  contracting  office 
is  individually  logged.  In  most  cases,  the 
process  flow  diagram  shows  the  median  start 
time  (i.e.,  one-half  of  the  cases  started  earlier 
and  one-half  later)  as  the  key  characteristic. 

In  a  few  instances,  where  it  seems 
sensible  to  do  so,  we  also  show  a  range  of  times 
for  various  sample  items  on  the  diagram  by 
using  a  shaded  area  to  the  left  and  to  the  right 
of  the  activity  block.  In  that  case,  the  left  end 
would  show  the  date  the  first  sample  item 
passed  through  that  activity  and  the  right  end 
would  show  the  date  for  the  last  item  in  the 
sample  The  median  time,  with  the  other 
information  for  the  step,  is  still  shown  in  the 
activity  block  within  the  shaded  area. 

When  the  complete  process  flow  has 
been  diagramed,  the  people  who  provided 
information  are  asked  once  again  to  review  the 


diagram  to  be  sure  that  it  accurately  repre¬ 
sents  the  actual  process.  Once  its  accuracy  is 
validated,  the  diagram  provides  the  basis  for 
analysis  and  streamlining. 

ANALYZING  THE  OPERATION  USING  PROCESS 
FLOW  TECHNIQUES 

One  of  the  immediate  results  of 
generating  a  flow  diagram  is  that  the  partici¬ 
pants  in  the  process  can  see  how  it  actually 
works  as  opposed  to  how  they  thought  it 
worked.  For  managers  somewhat  removed 
from  day-to-day  operations,  the  differences 
may  be  especially  revealing.  For  this  reason, 
the  diagram  is  an  invaluable  tool  for  analyzing 
and  streamlining  the  process,  particularly 
where  inter-divisional  chains  of  responsibility 
are  involved.  “There  are  several  methods  for 
prioritizing  work  processes  in  order  to  identify 
those  in  which  improvements  can  have  the 
greatest  impact. ”[5|  Some  of  these  methods 
are  matrix  analysis,  the  prioritization  grid,  or 
a  pareto  diagram. 

In  one  organization,  the  process  flow 
diagram  showed  a  long  period  of 
program/project  office  inactivity  preceding  the 
initiation  of  the  planning  process  for  the  next 
fiscal  year’s  contracting  activities.  The 
organization’s  explanation  was  that  no 
planning  is  undertaken  until  an  official 
memorandum  from  headquarters  is  received 
requesting  that  planning  begin.  We  found  no 
reason  why  the  program  or  project  office 
shouldn’t  start  earlier,  in  anticipation  of 
receipt  of  the  memorandum,  especially  since 
the  memorandum  is  usually  issued  later  than 
scheduled.  The  harm  caused  by  headquarter’s 
lateness  in  issuing  the  memorandum  and  by 
the  program  office’s  waiting  for  it  was  not 
realized  until  the  process  flow  diagram 
highlighted  the  adverse  impact. 

Where  time  standards  have  been 
developed  for  steps  in  the  award  cycle,  as  is 
frequently  the  case  in  the  contracting  office, 
the  team  can  compare  the  actual  process  time 
with  the  time  standard  for  completing  a 
specific  step.  It  can  also  determine  whether 
unplanned  delivery  of  upstream  documents 
(such  as  purchase  requests)  inhibits  the 
achievement  of  uniformity  in  workload 
throughout  the  year.  By  analyzing  the  process 
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flow  diagram  -  and  by  identifying  the  law, 
regulation,  policy,  or  practice  mandating 
various  steps  -  the  team  can  determine  why  a 
specific  individual  or  organization  is  involved 
in  one  or  more  steps.  It  can  also  start  the 
streamlining  process  by  identifying  steps  that 
are  being  performed  sequentially  when  it 
would  be  more  efficient  to  perform  them 
concurrently. 

Finally,  the  team  may  determine  the 
extent  to  which  a  step  adds  value,  the  nature  of 
the  value  added,  and  the  additional  adminis¬ 
trative  time  it  requires.  A  key  question  that 
may  be  asked  in  determining  the  value  of  a 
process  step  is;  How  much  extra  would  a 
customer  pay  to  have  that  step  included,  on  the 
basis  of  its  benefit  to  the  overall  process?!  51 

If  the  step  does  not  add  value  from  the 
customer’s  point  of  view,  it  should  be  elimi¬ 
nated  if  possible.  The  need  for  proper  iden¬ 
tification  of  the  specific  customer  for  each  step 
in  the  process  is  an  important  element  in 
process  flow  analysis.  However,  a  step 
required  by  law  (e.g.,  synopsizing  in  the 
Commerce  Business  Daily)  may  be  of  value  to 
the  general  public  but  be  little  appreciated  by  a 
specific  project  officer.  Identification  of  not 
only  the  customer,  but  of  the  customer’s 
requirements  is  an  essential  ingredient  of 
process  flow  analysis,  but  some  steps  must  be 
carried  out  regardless  of  whether  customers 
believe  they  add  value. 

A  difficult  requirement  of  good 
management  is  the  need  to  establish  measure¬ 
ments,  such  as  processing  time  standards,  that 
can  reasonably  be  applied  during  the  routine  of 
working  through  the  process.  The  process  flow 
diagram  is  helpful  for  identifying  where  such 
measurements  should  be  applied.  It  may  be 
used  to  highlight  natural  “gates”  that  can  be 
used  as  measurement  points,  or  it  can  suggest 
other  convenient  measures  to  monitor  perfor¬ 
mance. 

From  this  information,  we  are  then  in  a 
position  to  determine  which  steps  can  and 
should  be  modified,  consolidated,  or  elimi¬ 
nated. 

Once  process  flow  steps  have  been 
documented  at  several  different  organizations 


for  the  same  function  (e.g.,  acquisition 
processing),  it  becomes  clear  that  many  of  the 
steps  are  similar,  particularly  when  viewed  at 
higher  levels  of  aggregation.  On  the  basis  of 
this  observation,  it  is  then  possible  to  identify 
the  best  practice  applicable  to  each  step 
included  in  the  general  process  flow  diagrams, 
which  now  can  be  used  to  develop  a  “best 
practices”  model.  Organizations  can  then  com¬ 
pare  their  own  process  flow  diagrams  to  the 
“best  practices”  model  and  evaluate  the  value 
added  by  various  steps,  some  of  which  may  be 
different  from  those  in  the  model.  Such 
comparisons  may  also  result  in  revising  the 
“best  practices”  model. 

ONGOING  INITIATIVES  REQUIRING  PROCESS 
FLOW  DIAGRAMS 

Among  ongoing  initiatives  that  depend 
heavily  on  the  use  of  documented  process  flow 
diagrams,  we  take  note  of  three.  First,  such 
diagrams  are  a  basic  step  in  efforts  to  improve 
quality,  whether  known  as  TQM,  as  Quality 
Improvement,  or  by  some  other  name.  These 
efforts  are  well  documented  in  the  literature 
and  need  only  be  mentioned  here.  The  second 
is  the  DoD  Comptroller’s  performance 
measures  initiatives  described  earlier  in  this 
paper;  it  is  discussed  below.  The  third  is  an 
initiative  of  the  Governmental  Accounting 
Standards  Board,  which  is  also  discussed 
below. 

The  Comptroller  has  required  the  use  of 
work  process  tools  and  measures  to  support 
achievement  of  output  goals.  As  noted,  work 
process  measures  contribute  to  improving 
output  efficiency  or  effectiveness.  Some  work 
process  measures  complement  other  measures. 
The  Comptroller  notes  that  flow  charting  of 
specific  processes  may  be  useful  in  measuring 
performance,  as  is  work  force  and 
organizational  assessment.! 8 1 

The  comptroller  identified  a  number  of 
steps  to  be  taken:  facilitation  of  the  use  of 
work  process  improvement  tools,  systematic 
identification  and  codification  of  work  process 
tools,  identification  of  key  work  process  mea¬ 
sures,  updating  of  work  process  measures,  and 
training  and  educating  people  on  the  roles  and 
applications  of  work  process. |8 1 
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“Reducing  cycle  time  to  reduce  the 
resources  invested  in  the  production  pipeline 
over  the  production  cycle,  is  closely  correlated 
with  total  cost  per  output.  It  is  not  anticipated 
that  a  separate  cycle  time  indicator  will  be 
included  as  an  efficiency  measure  in  budget 
documents.  However,  on-time  performance 
will  be  recommended  as  an  effectiveness 
measure.  General  categories  of  output 
effectiveness  measures  include  quantity, 
timeliness,  quality,  and  customer  satis- 
faction.”[81 

Focus  on  the  concepts  for  improving 
delivery  of  services  is  certainly  not  confined  to 
DoD  or  to  the  Federal  Government.  To  develop 
a  broader  discussion  of  the  information 
necessary  to  assess  the  performance  of  service 
providers,  the  Governmental  Accounting 
Standards  Board  is  seeking  comment  on  efforts 
to  measure  outputs  in  the  provision  of  Govern¬ 
ment  services.  This  is  a  step  toward  a  State¬ 
ment  of  Governmental  Accounting  Concepts. 

There  is  a  clear  link  between  the  needs 
of  the  overall  operation  of  government  (public 
administration)  in  making  decisions  and  the 
needs  of  the  citizenry  in  assessing  the  account¬ 
ability  of  the  administrators.  Traditional 
governmental  financial  reporting  has 
attempted  to  provide  accountability  and 
decisionmaking  information  primarily  about 
fiscal  stewardship.! 3 1 

The  Governmental  Accounting 
Standards  Board  cites  three  separate  aspects  of 
operations  that  must  be  considered:  economy, 
efficiency,  and  effectiveness.  These  objectives 
can  be  better  achieved  if  management 
understands  the  organization’s  process  flow. 

CONCLUSIONS 

Even  well-run  acquisition  organizations 
with  long  and  successful  operating  histories 
are  unlikely  to  have  process  flow  diagrams 
documenting  the  steps  in  the  acquisition  pro¬ 
cess.  But  without  a  documented  process  flow, 
staffs  are  left  to  follow  the  undocumented 
process  by  rote.  Lack  of  a  documented  process 
flow  can  have  an  adverse  impact  upon  an 
organization’s  ability  to  make  quality  awards 
in  a  timely  manner.  This  is  particularly  true 
when  processes  change  fairly  rapidly  or  when 


individuals  leave  or  move  within  the  office, 
and  as  outside  circumstances  affect  the 
organization’s  mission  and  responsibilities. 
Yet,  in  spite  of  the  many  good  reasons  for 
having  a  well-documented  flow,  documen¬ 
tation  of  the  various  steps  in  the  process  or  of 
the  various  activities  is  seldom  required  or 
created. 

Without  a  documented  process  flow 
already  in  place,  it  is  time-consuming  and 
expensive  to  diagram  the  existing  procedures. 
But  because  process  flow  is  the  cornerstone  on 
which  so  many  improvement  techniques  are 
based,  the  effort  is  necessary. 

Perception  seldom  matches  reality  in 
acquisition  process  flow.  The  steps  actually 
being  performed  may  not  be  those  that  the 
participants  or  managers  think  are  being 
performed.  As  a  result,  decisions  are  often 
based  on  intuition  rather  than  on  fact.  Lack  of 
documented  process  flow  often  spawns  a 
breakdown  in  organizational  communication, 
resulting  in  finger-pointing  and  mistrust.  This 
situation  is  further  exacerbated  without 
adequate  monitoring  and  tracking  systems 
that  identify  the  various  steps  and  those 
responsible  for  them. 

Without  a  documented  process  flow, 
standard  processing  measurements  ( e.g.,  time), 
are  unavailable  for  most  steps  in  the  overall 
acquisition  cycle,  except  perhaps  for  the 
contracting  phase  itself.  This  portion  of  the 
process  is  the  one  most  likely  to  have  regularly 
published  standard  steps  and  processing  times. 

While  many  steps  are  similar  from  one 
organization  to  another,  with  only  a  small 
percentage  unique  to  a  specific  organization, 
few  attempts  have  been  made  to  standardize 
the  process  flow  from  organization  to  organi¬ 
zation. 

We  found  through  process  flow  analysis 
that  there  is  inadequate  linkage  between  the 
program  planning  and  budgeting  process  We 
also  found  no  obvious  impediment  to  starting 
the  planning  process  earlier;  there  is  no  need  to 
delay  it  until  the  budget  has  been  submitted  to 
Congress,  since  each  current  year’s  appropria¬ 
tion  has  a  strong  correlation  with  the  previous 
year’s.  Therefore,  the  project  planning  process 
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can  begin  with  the  initial  formulation  of  the 
budget  and  continue  as  the  budget  process 
progresses. 

In  addition  to  the  benefits  of  process  flow 
diagraming  and  analysis  that  have  been  dis¬ 
cussed,  the  following  advantages  may  also 
result: 

•  Evaluators  can  compare  reality  with 
perceptions 

•  A  baseline  can  be  established 

•  The  document  can  be  used  for  training 
and  educating  new  employees 

•  Project  officers  can  better  estimate  the 
total  processing  time,  including  all  of 
the  “downstream”  steps 

•  Contracting  officers  can  gain  apprecia¬ 
tion  for  all  of  the  “upstream”  steps  that 
must  occur  before  they  see  the  purchase 
request 

•  Management  can  better  estimate  the 
impact  of  new  programs  on  an  estab¬ 
lished  baseline. 

This  paper  has  described  the  work  that 
LMI  has  done  in  the  area  of  process  flow 
documentation  and  analysis  and  has  attemp¬ 
ted  to  suggest  the  benefits  of  documenting 
process  flows.  The  continuing  need  for  better 
public  administration,  as  well  as  the  require¬ 
ments  posed  by  new  initiatives,  make 
documented  process  flow  diagraming  and 
analysis  even  more  important  now  than 
previously. 
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ABSTRACT 

The  Malcolm  Baldridge  National 
Quality  Award's  (MBNQA) 
strength  is  the  criteria  and 
review  process  used  to  assess 
an  organization's  quality  im¬ 
provement  process.  Organiza¬ 
tion  Development  (OD)  techniques 
can  be  an  integral  part  of  the 
process  an  organization  uses 
for  achieving  excellence  based 
on  the  MBNQA  criteria.  Top 
leadership  is  the  key  factor 
for  success  using  either  strat¬ 
egy  for  organizational  change. 
They  must  create  a  climate 
which  supports  and  reinforces 
employee  motivation  to  perform 
quality  work.  Leaders  must  be 
involved  and  visible,  articu¬ 
late  values,  and  demonstrate 
consistency.  This  paper  de¬ 
scribes  the  value  of  linking 
these  powerful  and  influential 
concepts  for  increased  organi¬ 
zational  effectiveness  and  qua¬ 
lity  improvement. 


INTRODUCTION 

Total  quality  is  a  key  factor 
related  to  an  organization's 
productivity  or  to  the  level  of 
services  delivered.  Since 
1988,  the  Malcolm  Baldridge 
National  Quality  Award  (MBNQA) 
has  been  an  constructive  and 


influential  initiative  fcr  US 
industry  to  regain  its  competi¬ 
tive  edge.  The  strength  of 
the  MBNQA  is  the  set  of  crite¬ 
ria  and  the  review  process  used 
to  assess  the  effectiveness  and 
success  of  an  organization's 
quality  improvement  process. 
Criteria  cut  across  functional 
and  organizational  elements. 

Organization  Development  (OD) 
seeks  to  improve  organization 
effectiveness  using  techniques 
that  have  been  evolving  since 
the  late  1950 's.  For  maximum 
impact,  OD  efforts  need  to  be 
systematic  and  planned  for  the 
entire  organization.  Behavio¬ 
ral  science  interventions  in 
the  organization's  processes 
should  be  designed  to  align 
organization  and  business  ob¬ 
jectives.  The  process  must  be 
supported  from  the  top  down  to 
maintain  the  organization's 
vitality  and  competitiveness. 
Because  of  the  behavioral  na¬ 
ture  of  most  OD  interventions, 
collaborative  work  efforts  are 
also  enhanced.  These  tech¬ 
niques  often  amplify  individual 
potential  and  sense  of  self- 
worth  which  also  has  an  added 
value  for  the  organization. 

EVOLUTION  OP  TQM' 

Before  World  War  II,  the  War 


389 


Department  saw  a  need  to  con¬ 
trol  the  quality  of  materials 
and  manufactured  products.  in 
December  1940  the  American 
Standards  Association  was  asked 
to  begin  a  project  to  apply 
statistics  to  the  quality  of 
materials  and  products  to  be 
used  in  the  war  effort.  Three 
classified  standards  were  de¬ 
veloped  and  used  by  the  allies. 
These  standards  addressed  prod¬ 
uct  and  parts  sampling  and  pro¬ 
cess  control  charts. 

Shewhart  did  much  of  the  early 
work  describing  process  varia¬ 
tion  in  descriptive  terms.  He 
found  that  all  types  of  repeat- 
able  activities,  from  manufac¬ 
turing  to  administrative  pro¬ 
cesses,  are  characterized  by 
variation.  This  lead  to  his 
recognition  of  the  importance 
of  being  able  to  monitor  these 
processes  to  detect  changes. 

During  the  fifties,  American 
industry  could  not  make  prod¬ 
ucts  fast  enough  to  meet  the 
pent  up  demand.  Quality  con¬ 
trol  learned  and  required  for 
the  war  effort  was  secondary  to 
sales  and  production.  In  Ja¬ 
pan,  however,  statistical  qual¬ 
ity  control  charts  were  being 
used  to  monitor  variability  and 
product  acceptability.  With 
quality  as  a  strategic  priori¬ 
ty,  the  Japanese  realized  that 
if  a  process  resulted  in  de¬ 
fects,  even  the  best  inspection 
methods  would  not  eliminate  the 
problem.  They  noted  that  prod¬ 
ucts  made  without  defects  pro¬ 
duced  significant  savings  which 
were  more  cost  effective  than 
inspection  or  defect  repair. 

There  were  still  US  companies 
using  technical  and  statistical 
tools  to  ensure  product  quality 
in  the  sixties.  Major  com¬ 


panies,  especially  those  who 
were  involved  in  the  war  ef¬ 
fort,  had  formal  quality  and 
reliability  organizations  that 
practiced  statistical  quality 
control  along  with  design  of 
experiments  and  reliability 
engineering.  Over  the  decade, 
these  methods  began  to  fade  as 
the  United  States  continued  to 
experience  phenomenal  market 
growth.  Quality  took  a  back 
seat  to  delivery. 

American  market  growth  contin¬ 
ued  through  the  seventies.  At 
the  same  time,  Japan’s  world 
competitiveness  began  to  take 
hold.  Japanese  products  began 
taking  over  market  share  be¬ 
cause  they  gave  consumers  supe¬ 
rior  quality  for  less  money. 

How  did  the  United  States  lose 
so  many  product  markets  to  the 
Japanese  in  such  a  short  time? 

•  In  the  fifties,  the  Japan¬ 
ese  started  to  institutionalize 
the  delivery  of  quality  prod¬ 
ucts  and  services 

•  All  management  levels  rec¬ 
ognized  that  quality  of  product 
or  service  not  only  results  in 
increased  customer  demand,  but 
is  cost  competitive 

•  All  employees  are  expected 
to  participate  in  any  activity 
that  can  increase  effectiveness 

•  Employee  contributions  are 
sought  out  and  recognized  by 
management 

•  Teamwork  throughout  an  or¬ 
ganization  is  essential  and  a 
core  value 

•  The  quarterly  statement  is 
not  the  bottom  line,  rather  it 
is  to  make  the  enterprise  "re- 


cession  proof"  (The  CEO  of 
Ricoh  Co.,  LTD.,  1975  recipient 
of  the  prestigious  Deming 
Award,  stated  "...  to  make  our 
company  recession  proof,  with 
true  sale  and  technology  capa¬ 
bilities  .  " )  ^ 

Attitudes  in  the  United  States 
began  to  change  during  the 
eighties.  One  catalyst  was  the 
NBC  white  paper,  "If  Japan  Can 
. . .  Why  Can't  We?"  Six  months 
after  the  broadcast.  Ford  Motor 
Company  contacted  Deming  for 
help  using  quality  improvement 
as  a  competitive  strategy. 

Deming 's  major  thrust  in  the 
40 's  was  statistical  process 
control  (SPC) ;  however,  by  1980 
his  major  emphasis  had  turned 
to  managerial  matters.  Deming 
wanted  to  "create  a  prairie 
fire  that  would  consume  all 
America  and  turn  it  around."-' 
Dr.  Deming' s  14  points  are  a 
prescription  for  a  company's 
competitive  success.  In  his 
seminars,  he  is  continually 
telling  American  management 
they  are  responsible  for  ap¬ 
proximately  85  percent  of  the 
problems  faced  in  industry  and 
business . 

Work  with  Deming  led  Ford  to 
form  a  multifunctional  team 
consisting  of  approximately  100 
people  who  moved  in  and  out  of 
the  team  (called  the  Alpha 
Team)  on  a  rotating  basis. 
Education  of  the  entire  cor¬ 
poration  on  quality  and  coop¬ 
eration  was  important,  and  ex¬ 
tensive  time  was  spent  develop¬ 
ing  values  and  guiding  princi¬ 
ples  . 

Although  you  couldn't  prove  it 
by  sales  of  books  espousing 
successful  Japanese,  not  Ameri¬ 
can,  management  techniques. 


industry  and  business  leader¬ 
ship  is  changing.  We  now 
expect  two  way  vertical 
communication  between  manage¬ 
ment  ,  and  have  employees  par¬ 
ticipate  on  teams  that  are  al¬ 
lowed  to  tell  management  how 
organizational  performance  can 
be  improved . 


EVOLUTION  OP  ORGANIZATION 
DEVELOPMENT 

Organization  Development  im¬ 
plies  change.  Traditionally, 
OD  efforts  are  facilitated  by 
individuals  with  behavioral 
science  backgrounds,  rather 
than  business  or  engineering 
backgrounds.  OD  techniques  and 
consultants  are  useful  for  man¬ 
agers  seeking  to  plan  and  im¬ 
plement  change,  and  enhance  the 
potential  for  individual  and 
organizational  growth  through  a 
balanced  concern  for  people  and 
for  organization  performance. 

When  interventions  occurred  at 
the  top  of  the  organization, 
there  was  often  a  lack  of  con¬ 
gruence  between  what  the  execu¬ 
tives  said,  and  what  they  did. 
In  today's  TQM  jargon  they 
"didn't  walk  their  talk."  Of¬ 
ten,  the  top  leaders  were  over 
dependent  on  outside  help,  and 
sought  to  accomplish  signifi¬ 
cant  changes  in  too  short  of  a 
period.  For  change  to  have  a 
lasting  impact,  it  must  cut 
deeply  into  the  organization's 
structural  processes  and  sys¬ 
tems,  and  lead  to  the  modifica¬ 
tion  of  employee  attitudes  and 
motivations  through  empower¬ 
ment,  participation,  trust,  and 
cooperat ion . 

Mayo's  work  at  the  Hawthorne 
plant  of  Western  Electric  in 
the  1930s  set  the  stage  for  the 
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human  relations  school  of  man¬ 
agement.  During  the  forties, 
Lewin ' s  "field  theory"  provided 
the  foundation  for  seeking  more 
effective  organizational 
change.  His  three  steps  (un¬ 
freezing,  movement,  and  re¬ 
freezing)  ,  while  simple,  are 
very  powerful  when  combined 
with  analytic  methods  such  as 
force  field  analysis. 

Despite  the  name,  much  of  the 
early  practice  of  organization 
development  was  at  the  work 
group  level,  and  relied  on  ex¬ 
perience  based  activities. 
Laboratory  Training  (T-groups) 
were  recognized  as  a  means  to 
affect  individuals  and  work 
teams.  The  intent  of  these 
activities  was  to  change  the 
organization.  McGregor's  "The¬ 
ory  X"  and  Maslow's  description 
of  a  needs  hierarchy  ■  pped  by 
self  actualization  prci/ided  a 
viable  scheme  to  help  managers 
understand  their  work  force 
during  the  late  fifties.  Mc¬ 
Gregor  applied  laboratory 
training  to  large  systems  (Un¬ 
ion  Carbide) ,  and  Blake  and 
Shepard  offered  their  T-group 
services  to  improve  relation¬ 
ships  between  top  managers. 
The  University  of  Michigan's 
Survey  Research  Center  was  es¬ 
tablished  and  used  a  model  of 
diagnosis,  feedback,  and 
action . 

Many  OD  consultants  are  only 
concerned  with  the  human  pro¬ 
cesses  found  in  organizations, 
despite  the  fact  that  Trist 
described  organizations  as  a 
sociotechnical  system  starting 
in  the  1960s.  His  work  lead  to 
the  idea  of  open  system  plan¬ 
ning  in  which  it  was  essential 
to  recognize  that  the  organiza¬ 
tion  is  affected  by  its  envi¬ 
ronment.  This  was  further  en¬ 


hanced  by  Katz  and  Kahn's  mul¬ 
tidimensional  social  system. 
Likert ' s  strong  advocacy  of 
participative  management  and 
system  level  thinking  based  on 
his  four  models  for  organiza¬ 
tional  design  and  the  concept 
of  managers  as  linking  pins 
between  organization  levels 
also  added  to  the  practice. 
Managerial  Grid  surfaced  as  a 
development  concept,  and  stra¬ 
tegic  planning  took  on  greater 
importance . 

During  the  seventies.  Manager¬ 
ial  Grid  became  one  of  the  most 
popular  OD  techniques.  It  was 
often  linked  to  strategic  im¬ 
plementation,  management  by 
objectives,  and  ultimately  to 
organizational  plans.  Hackman 
and  Oldham  emphasized  job  sat¬ 
isfaction,  while  Herzberg  de¬ 
scribed  motivating  conditions. 
By  the  middle  1970s,  Weisbord^ 
proposed  a  "six  box"  model  for 
organizational  diagnosis. 
Weisbord’s  model  also  contained 
provisions  for  assessing  the 
organization's  environmental 
interface.  The  six  boxes  are: 

PURPOSE  -  The  business  or  mis¬ 
sion  of  the  organization 

STRUCTURE  -  How  work  is  divided 

RELATIONSHIPS  -  How  people  work 
together,  manage  conflict,  and 
use  technology 

REWARDS  -  How  work  tasks  are 
incentivized 

HELPING  MECHANISMS  -  Techniques 
and  technologies  which  allow 
activities  to  be  coordinated 

LEADERSHIP  -  The  per<~op  cr  pco 
pie  who  keep  the  first  five 
boxes  in  balance 
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In  the  1980 's  both  worker  (mo¬ 
tivation)  and  customer  expecta¬ 
tions  (quality)  were  changing. 
Argyris  and  Schon  described 
learning  in  organizations,  and 
leadership's  use  of  espoused 
theory  and  practice  of  manage¬ 
ment  (do  they  do  what  they 
say?) .  Others  advocated  the 
concept  of  strategic  vision. 
These  changes  required  a  refo¬ 
cus  of  management  attention  and 
techniques.  As  you  can  see, 
many  approaches  by  many  behav¬ 
ioral  scientists  permeated  US 
industry  during  these  decades. 


ORGANXZATXON  DEVELOPMENT  TODAY 

Contemporary  organization  de- 
vf'Topment  techniques  along  with 
their  inherent  objectives  now 
recognize  that: 

•  Management  must  be  sensi¬ 
tive  to  customer  expectations. 

•  Product  and  service  avail¬ 
ability  does  not  mean  accepta¬ 
bility  . 

•  Management  doesn't  have  all 
the  answers  regarding  elements 
that  increase  an  organizations 
effectiveness . 

•  Employee  participation  is 
a  major  key  to  solving  produc¬ 
tivity  and  quality  problems. 

•  Quality  of  product  and  ser¬ 
vice  is  the  major  reason  (more 
important  than  cost)  for  pur¬ 
chase  decisions  (at  least  in 
the  commercial  sector) . 

•  The  direct  labor  cost  that 
was  the  major  thrust  of  manage¬ 
ment's  plans,  direction,  con¬ 
trolling,  etc.  has  shrunk  from 
the  30-50%  of  the  work  force  in 
the  60's  and  70's  to  the  10-15% 


range  in  most  industries  today. 

•  The  factors  that  affect  a 
company's  performance,  quality 
and  productivity  are  found  in 
all  segments  of  the  organiza¬ 
tion  --  marketing,  finance, 
personnel,  design,  engineering, 
procurement  and  manufacturing. 

Organization  Development  prac¬ 
titioners  believe  they  are 
change  agents.  In  combination 
with  a  systems  approach,  inter¬ 
nal  and  external  OD  consultants 
intervene  at  the  work  group 
level.  They  use  applied  behav¬ 
ioral  science  to  change  orga¬ 
nizations  and  their  processes. 
This  normally  includes  estab¬ 
lishing  on-going  normative  re- 
educative  strategy  for  change. 

Organizational  goals  are  their 
target.  The  goals  must  flow 
from  the  organization's  pur¬ 
pose,  and  may  be  etched 
throughout  the  organization 
with  vision  and  mission 
statements.  Top  leaders  of  the 
organization  must  create  an 
organization  climate  which  sup¬ 
ports  and  reinforces  employee 
motivation  to  work  toward  and 
achieve  these  goals  and  belief 
in  quality  (which  may  seem  dis¬ 
tant  or  nebulous  from  the  low¬ 
est  organizational  levels.) 

Collaboration  was  highly  valued 
by  the  OD  practitioner.  Many 
efforts  had  limited  impact 
since  teams  were  involved  in 
working  things  out,  rather  that 
asking  senior  executives  to 
clarify  goals  or  establish  pri¬ 
orities.  These  teams  also 
failed  to  communicate  to  top 
management  their  views  of  where 
executives  attention  should  be 
focused . 

Organization  goals  can  be 
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achieved  by  diagnosing  the 
current  situation,  taking  ac¬ 
tion,  and  reassessing  the  prog¬ 
ress.  This  often  results  in  a 
change  of  process  as  shown  in 
Figure  1.  Individuals  in  the 
organization  are  expected  to 
learn  from  the  experience  (they 
are  part  of  action  based 
research  where  both  they  and 
the  consultant  are  part  of  the 
"experiment.")  The  five  areas 
which  follow  are  typical  of 
those  explored  during  the  diag¬ 
nosis  phase. 


Figure  1 

Diagnostic  Action  Model 


•  ConununjLcatlons :  Who  talks 
to  whom,  how  often,  and  about 
what?  Do  the  communications 
fit  the  nature  of  the  jobs? 
Observation,  surveys,  inter¬ 
views,  focus  groups,  or  analy¬ 
sis  of  audio  or  video  tape  are 
used  to  gather  the  data. 

*  Leaderahip/Boss  Subordinate 
Relatlonahlpa ;  How  do  the  bos¬ 
ses  relate  and  lead/manage? 
What  is  their  style  and  its 
impact?  Assessment  can  be 
accomplished  through  question¬ 
naires,  interviews,  and  obser- 
vat ion . 


9  Oeclalon  Making  and  Goal 
Setting:  Who  does  it?  Who 
participates  and  how?  Do  they 
have  the  skills?  Do  they  have 
both  short  and  long  range  in¬ 
sights  and  temperaments?  Data 
can  be  gathered  by  observation, 
and  consequence  analysis. 

•  Conflict  Resolution:  Where 
do  the  conflicts  occur?  What 
methods  are  used?  Does  the 
organization  have  a  formal  pro¬ 
cess?  How  are  they  managed? 
What  is  the  impact  of  the  re¬ 
ward  system  on  conflict?  In¬ 
terviews  and  observation  are 
the  best  way  to  get  at  the 
heart  of  this  area. 

9  Managing  the  Interface: 

How  do  two  parties  get  along? 
This  can  be  a  subset  of  con¬ 
flict  resolution,  and  ties  into 
the  goals  of  the  organization 
and  its  subordinate  units. 
Thus,  the  need  for  consistency. 
Interviews,  surveys  and  obser¬ 
vation  are  the  best  means  to 
determine  impact . 


THE  BAIDRXGE  PROCESS 

So  that  our  competitiveness 
would  not  be  completely  lost, 
the  country  needed  to  influence 
and  support  business.  This 
concern  resulted  in  Congress¬ 
ional  enactment  of  the  Malcolm 
Baldrige  National  Quality  Award 
(MBNQA)  into  legislation  in 
1987.  The  purpose  was  to  fos¬ 
ter: 

•  Awareness 

•  Recognition 

•  Information  transfer 

Whether  some  detractors  think 
it  or  not,  the  most  effective 
response  to  global  competitive¬ 
ness  is  the  MBNQA.  Many  states 
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and  countries  have  adopted  the 
award.  It  is  used  by  companies 
applying  for  the  national  award 
and,  more  frequently,  for  self- 
assessment  for  continuous  im¬ 
provement.  Manufacturing,  ser¬ 
vice  and  small  business  compa¬ 
nies  can  apply  for  the  award. 
Whether  applying  for  the 
national  award  or  for  self  as¬ 
sessment,  the  process  is  gener¬ 
ally  the  same. 

A  description  of  the  company's 
quality  management  system  is 
recorded  in  response  to  the 
MBNQA  application  guidelines. 
Guidelines  cover  seven  major 
categories,  twenty-eight  sub¬ 
categories  and  a  total  of  nine¬ 
ty-two  specific  items.  Major 
categories  and  point  allocation 
are : 

Leader ahlp  (95) 

Information  and  Analysis  (75) 

Strategic  Quality  Planning  (60) 

Human  Resource  Development  and 
Management  (150) 

Management  of  Process  Quality 

(40) 

Quality  and  Operational  Results 

(180) 

Customer  Focus  and  Satisfaction 

(300) 

The  company's  quality  manage¬ 
ment  system  description  is  re¬ 
viewed  by  a  group  of  examiners. 
They  note  the  strengths  and 
areas  needing  improvement  for 
all  92  items.  Scores  are  then 
recorded  for  the  sub-categories 
based  on  the  strengths  and 
wearinesses . 

In  the  national  process,  the 


examiner's  reports  are  evaluat¬ 
ed  by  judges.  Based  on  scores 
and  comments,  the  judges  may 
refer  the  application  bac)c  to 
the  examiners  for  another  re¬ 
view.  This  second  phase  review 
requires  consensus  for  areas  of 
significant  examiner  deviations 
on  the  scores  or  comments. 
During  the  second  review,  they 
also  indicate  the  site  visit 
issues  for  verification  and 
clarification  of  data  present¬ 
ed.  A  site  visit  is  the  last 
step  for  the  national  award 
finalist.  This  process  is  the 
model  (see  Figure  2)  used  by 
most  companies  for  self-assess¬ 
ment  . 


Figure  2 


Evaluation  Process 
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Table  1 


Areas  o£  Wealuiess 


ORGANIZATIONAL  DEVELOPMENT 

BALDRIGE  PROCESS 

Non-technical/Sof t  science 

Non-prescript i ve 

Non-tangible  bottom  line 

Concentration  on  quality  im¬ 
provement  clouds  bottom  line 

Organizational  effectiveness 
undefined 

Written  material  is  the  major 
assessment  vehicle 

Intrusion  of  outside  consul¬ 
tants  into  the  organization 

Difficult  for  a  company  to 
think  and  speak  in  unity 

Client  system  is  usually  a 
small  part  of  the  organization 

Disregards  organizational 
structure 

The  journey  for  quality  im¬ 
provement  based  on  Baldrige 
criteria  most  often  has  signif¬ 
icant  impact  on  the  way  people 
are  treated,  empowered,  recog¬ 
nized  and  rewarded.  It  also 
impacts  the  manner  in  which 
management  leadership  is  prac¬ 
ticed,  and  how  productivity  and 
quality  (effectiveness)  is  im¬ 
proved  . 


A  SYNERGXSTJC  MERGER 

The  major  area  where  MBNQA  and 
OD  come  together  is  LEADERSHIP. 
While  we  have  been  noting  the 
historical  perspective  and 
techniques  of  these  approaches, 
we  should  note  some  shortcom¬ 
ings.  Table  1  above  depicts 
some  OD  and  MBNQA  areas  of 
weakness . 

When  American  firms  were  making 
extraodinary  levels  of  profits, 
industrial  leadership  was  fre¬ 
quently  made  the  prerogative  of 
financial  managers.  Leaders 


managed  for  growth,  with  little 
concern  for  improvement  of  pro¬ 
duct  and  services.  Business 
schools  advocated  a  general 
theory  of  management;  i.e.,  a 
good  manager  can  manage  any 
type  of  enterprise.  A  person 
need  not  know  anything  about 
steel  to  head  a  steel  company, 
have  railroad  experience  to 
manage  a  railroad  company,  etc. 

These  executives  and  managers 
were  action  oriented  and  wanted 
to  solve  their  own  problems. 
They  often  reached  the  top 
without  adequate  preparation  to 
handle  a  rapidly  changing  busi¬ 
ness  and  world  environment 
since  their  cultural  heritage 
was  based  on  quarter  by  quarter 
management .  They  were  the 
stars.  Yet  many  were  not  ready 
to  recognize  the  transforma¬ 
tions  which  were  needed  when 
threatened  with  price  and  qual¬ 
ity  competition,  or  new  market 
opportunities . 

If  we  agree  with  the  above  and 


keep  these  weaknesses  in  mind, 
OD  and  MBNQA  processes  can  com¬ 
plement  each  other.  Greater 
improvement  and  success  can 
result.  Leadership  is  the  com¬ 
mon  element  where  OD  and  Bal- 
drige  come  together  and  can 
illustrate  the  strength  of  this 
synergism. 

Leader's  seek  goal  attainment. 
They  need  to  be  able  to  fully 
mobilize  people  and  recognize 
the  need  for  congruence  between 
people,  process,  structure  and 
environment.  Highly  effective 
leaders  recognize  that  values 
can't  be  imposed.  Thus,  they 
know  it  is  essential  to  create 
an  organization  culture  in 
which  employees  share  the  same 
or  similar  goals  as  managers. 
The  MBNQA  can  provide  the  tar¬ 
get  for  both  to  focus  their 
energy . 

Customer  focus  and  satisfaction 
is  heavily  weighted  by  the  Bal¬ 
dridge  point  allocation.  With 
the  organization  paradigm 
shifting  toward  a  customer  fo¬ 
cus,  there  is  a  need  to  recog¬ 
nize  the  worker  may  be  caught 
in  the  middle.  If  the  customer 
desires  something,  how  respon¬ 
sive  should  the  employee  be  if 
his  or  her  boss  still  controls 
the  organization's  rewards? 

To  increase  the  liklihood  of  a 
successful  quest  for  the  top 
levels  of  the  Baldridge,  lead¬ 
ers  can  undertake  an  OD  pro¬ 
gram.  With  the  appropriate 
choice  of  process  and  interven¬ 
tions,  OD  can  build  trust  be¬ 
tween  organization  levels  or 
between  organization  functions; 
advocate  an  open  problem  solv¬ 
ing  climate;  relocate  decision 
making  and  problem  solving  to 
the  source  of  information;  in¬ 
creases  the  sense  of  individual 


ownership;  move  toward  greater 
collaboration;  and  increase 
awareness  of  group  process. 
The  basic  value  is  one  of  in¬ 
dividual  choice,  within  a  joint 
action  planning  framework. 
This  fits  nicely  into  the  con¬ 
cept  that  each  person  should  be 
responsible  for  quality. 

Such  a  program  blends  with  TQM 
which  seeks  to  align  work 
through  the  common  focus  of 
customer  satisfaction,  and  is 
accomplished  by  seeking  methods 
to  continually  improve  process¬ 
es,  products,  and  services. 
The  challenge  is  to  control 
variability  so  that  the  result 
falls  within  acceptable  limits. 
It  requires  the  entire  organi¬ 
zation  to  work  together  toward 
common  objectives.  Leaders 
must  be  involved  and  visible, 
articulate  values,  and  demon¬ 
strate  consistency.  Leaders 
would  do  well  if  they  learned 
and  used  OD's  process  consulta¬ 
tion  and  team  development  tech¬ 
niques  . 


CONCLUSION 

The  MBNQA  establishes  a  frame¬ 
work  for  a  systematic  appraisal 
and  diagnosis  of  quality  im¬ 
provement  system  problems. 
Quality  and  productivity  mea¬ 
surements  of  the  Baldrige  cri¬ 
teria  need  to  be  tied  into  OD's 
effectiveness.  Many  OD  tech¬ 
niques  can  be  an  integral  part 
of  the  process  organizations 
use  to  achieve  or  exceed  the 
Baldridge  criteria  since  these 
techniques  improve  communica¬ 
tions,  teamwork,  involvement, 
and  cohesiveness. 

Top  leadership  is  the  key  fac¬ 
tor  for  success  using  either 
strategy  for  organizational 
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change.  However,  the  implemen¬ 
tation  strategy  for  a  quality 
program  must  be  well  thought 
out  if  the  desired  success  is 
to  be  achieved.  With  the  mar¬ 
riage  of  quality  and  productiv¬ 
ity  goals  and  results  to  busi¬ 
ness  goals  and  results,  a  com¬ 
pany  can  address  the  Baldridge 
criteria  more  easily  and  more 
effectively.  Meeting  these 
criteria  can  be  reflected  in 
the  organization's  strategic 
business  plan. 
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ABSTRACT 

The  goal  of  establishing  a  professional 
acquisition  workforce  is  becoming  more  and 
more  important  as  the  number  and 
complexity  of  laws,  executive  orders,  and 
regulations  continues  to  increase  with  no  end 
in  sight.  This  goal  can  be  achieved  only 
through  a  concerted  team  effort  between 
Government,  Industry  and  Academe.  This 
team  effort  is  being  put  into  place:  The 
Department  of  Defense  (DoD),  with  the 
prodding  of  Congress,  has  instituted 
requirements  for  a  professional  acquisition 
workforce;  the  Office  of  Federal 
Procurement  Policy  (OFPP)  has  issued  a 
Policy  Letter  designed  to  levy  similar 
requirements  on  Civilian  Agencies;  the 
Federal  Acquisition  Institute  (FAI)  has 
published  the  long  awaited  Contract 
Specialist  Workbook:  there  is  an  increase  in 
the  number  of  colleges  and  universities 
offering  acquisition  courses  and  degrees; 
and,  there  is  an  increased  demand,  from  the 
private  sector,  for  educated  and  trained 
acquisition  professionals.  The  goal  appears 
to  be  within  view. 

This  paper  is  a  ten  year  update  to  a  paper 
presented  at  the  1983  Federal  Acquisition 
Research  Symposium.  The  paper  explores 
what  has  occurred  in  the  intervening  years 
to  establish  a  professional  acquisition 
workforce.  It  preseits  the  results  of  a 
replication  of  the  initial  study’s  review  of 
advertisements  in  newspapers  and  Contract 
Management  magazine.  Finally,  it  explores 


where  the  efforts  to  establish  a  professional 
acquisifon  workforce  are  headed  and  what 
still  remains  to  be  accomplished. 

INTRODUCTION 

In  June,  1990,  the  National  Contract 
Management  Association  stated,  "Keeping 
up  with  the  technological  changes  and 
meeting  the  challenges  of  the  1990’s  will 
require  an  educated,  experienced  workforce 
committed  to  a  high  degree  of  expertise, 
performance  excellence,  and  ethical 
standards  -  in  short,  professionals."  (5) 
Many  professions  establish  specific 
requirements  for  their  professionals; 
doctors,  lawyers,  certified  public 
accountants,  nurses,  real  estate  agents  and 
stockbrokers  all  have  educational/degree 
requirements,  certifications  and/or  testing 
requirements  in  order  to  work  in  their 
fields.  It  is  reasonable,  and  necessary,  to 
expect  that  for  acquisition  to  truly  become  a 
profession,  it  must  have  specific  education, 
training  and  experience  criteria  with 
accompanying  certification  requirements. 
The  theme  of  the  1993  Acquisition  Research 
Symposium,  "Acquisition  for  the  Future: 
Imagination,  Innovation,  and 
Implementation"  is  especially  relevant  to  the 
issue  of  acquisition  professionalism.  The 
goals  have  been  imagined,  now  the 
innovation  and  implementation  are 
necessary.  The  original  exploration  of  this 
topic  was  presented  at  the  1983  Federal 
Acquisition  Research  Symposium,  which 
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had  the  theme,  "Government,  Industry  and 
Academe:  Synergism  for  Acquisition 

Improvement."  (7)  As  was  stated  in  the 
previous  pap)er,  the  goals  cannot  be  reached 
without  true  cooperation  and  organization  of 
efforts  among  the  three  players: 
Government,  Industry  and  Academe.  Each 
element  of  this  necessary  "triumvirate"  has 
made  efforts  towards  the  goal  and  there 
appears  to  be  some  coordination  of  efforts. 
However,  stronger  teamwork  and  change  are 
still  needed.  Change  is  not  easy.  Just  as 
there  are  change  agents,  there  are  its  foes. 
As  Robert  F.  Kennedy  said,  "Progress  is  a 
nice  word.  But  change  is  its  motivator  and 
change  has  its  enemies."  (9)  Throughout 
this  paper  we  will  explore  both  these  change 
agents  and  the  enemies  of  change,  what  has 
been  achieved  and  what  is  left  to  be 
accomplished. 

GOVERNMENT 

In  a  March,  1991,  article.  Dr.  James  H. 
Edgar,  CPCM,  stated,  "The  qualifications, 
training,  and  personnel  management  of 
individuals  assigned  to  acquisition  positions 
have  been  matters  of  long-standing  concern 
to  Congress  as  well  as  to  the  executive 
branch."  (6)  The  Office  of  Federal 
Procurement  Policy  (OFPP)  was  established 
within  the  Office  of  Management  and 
Budget  (0MB)  in  1974  to  provide  overall 
direction  of  procurement  policies  and  to 
coordinate  programs  to  improve  the  quality 
and  performance  of  agency  procurement 
personnel.  (19)  OFPP  had  an  original 
authorization  for  a  five-year  period  and  was 
reauthorized  twice,  for  four  years  each  time. 
In  1988,  Congress  enacted  the  Office  of 
Federal  Procurement  Policy  Act 
Amendments  which  finally  established  OFPP 
permanently.  Clearly,  Congress  recognized 
that  establishing  a  professional  procurement 
workforce  was  going  to  be  a  long  and  drawn 
out  process. 


The  attempts  by  0PM  to  lower  grade  levels 
of  acquisition  personnel,  described  in  the 
previous  paper,  have  been  stopped.  The 
grade  levels  have  been  maintained.  In  fact, 
in  1989  0PM  reclassified  the  1102  series 
from  administrative  to  professional.  This 
was  a  great  step  forward  in  enabling  the 
establishment  of  a  professional  acquisition 
workforce.  But  we  must  ensure  that  we 
follow  through. 

Although  there  still  is  some  promotion  of 
clerical  and  administrative  personnel  into  the 
acquisition  field,  the  Government  is  bringing 
more  college  graduates  into  the  field.  FAI’s 
1992  report  identified  that  of  the  2,435  total 
hires  into  the  1102  series,  in  FY  1991, 
1,728  were  from  other  Government  fields. 
Although  many  may  have  been  clerical  or 
administrative  personnel,  40%  were  college 
graduates,  an  increase  of  7%  from  the 
previous  study.  And  52%  of  the  college 
graduates  had  degrees  in  business,  law  or 
public  administration.  Statistics  for  the 
degree  field  were  not  even  available  at  the 
time  of  the  previous  study.  Of  the  644 
recruited  from  outside  the  Government,  85  % 
had  college  degrees  (61%  in  business,  law 
or  public  administration).  In  1991,  the 
percentage  of  contract  specialists  with  a 
Bachelor’s  Degree  was  53%.  Sixty  percent 
of  the  1 102’s  with  degrees  majored  in 
business,  law  or  public  administration. 
While  these  numbers  may  be  impressive,  it 
cannot  be  shown  that  there  has  been  an 
increase  in  the  overall  number  of  1102’s 
with  Bachelor’s  Degrees  in  business  or 
business  related  fields.  The  reason  for  this 
inability  to  compare  statistics  is,  as 
Professor  Harry  Page  stated,  in  testimony  to 
a  Senate  subcommittee  in  1986,  describing 
education  levels  of  the  federal  procurement 
workforce  at  that  time,  "Perhaps  one  third 
are  college  graduates  and  no  one  knows  how 
many  of  these  have  actually  studied  business 
and  management  or  related  subjects 
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(emphasis  added)."  (12)  In  the  future,  we 
will  be  able  to  track  any  improvements  in 
this  areas.  The  FAI  study  found  a  positive 
relationship  between  lengthy  experience  in 
the  1 102  series  and  the  likelihood  of  having 
a  Bachelor’s  Degree.  Sixty-one  percent  of 
the  personnel  who  had  been  in  the  1102 
series  at  least  15  years  (at  the  time  of  the 
1991  study)  had  Bachelor’s  Degrees.  It  is 
possible  that  the  additional  years  of  service 
gave  personnel  more  time  to  earn  degrees. 
(18) 

To  add  to  the  somewhat  promising  statistics 
on  education  and  degrees,  FAI  reported  that 
FY  1991  had  the  lowest  loss  rate  of  1102’s 
since  its  reporting  started  in  FY  1978, 
resulting  in  personnel  having  more 
experience  in  the  field.  Fifty-six  percent  of 
the  1102’s  had  more  than  six  years 
experience  in  the  series.  But  surely  that  is 
insufficient  for  a  truly  professional 
workforce.  These  statistics  show  some 
progress  is  being  made,  but  our  inability  to 
retain  people  for  longer  periods  does  not 
bode  well  for  continuity  or  quality. 
Unfortunately,  a  major  potential  solution  to 
retention,  higher  pay,  is  the  deadly  enemy 
of  deficit  reduction. 

0MB’ s  Policy  Letter  92-3,  entitled, 
"Procurement  Professionalism  Program 
Policy  -  Training  for  Contracting 
Personnel,"  affirmed  that  "The  quality  of 
contracting  actions  depends  largely  on  the 
professional  skills  of  the  Government 
procurement  workforce  to  help  meet  agency 
mission  needs."  (14)  This  Policy  Letter 
established  a  Government- wide  training 
standard  for  skill-based  training  in 
performing  contracting  duties. 

Janice  M.  Menker,  CPCM,  of  the  Defense 
Systems  Management  College,  stated, 
"Education  and  professionalism  do  not  come 
cheap.  As  federal  and  state  budgets  decline. 


government  must  do  more  with  less.  To 
which  does  the  government  give  priority  - 
education  or  goods  and  services?  Industry 
must  also  make  a  choice  -  invest  in 
education  and  training  or  maintain 
competitiveness  in  the  market.”  (10)  The 
cost  of  the  education  of  the  acquisition 
workforce  must  be  considered  in  light  of  the 
magnitude  of  the  cost  of  the  goods  and 
services  that  the  workforce  purchases  and 
the  benefits  to  be  achieved  by  such  training. 
"In  1991,  over  31, OCX)  Federal  procurement 
professionals  were  responsible  for 
purchasing  more  than  $191  billion  in  goods 
and  services  for  the  Federal  Government." 
(20)  The  quality  of  workers  responsible  for 
contracting  and  procurement  actions  has 
come  under  close  scrutiny.  The  growing 
budget  deficit  and  allegations  of  waste  and 
mismanagement  have  contributed  to  this 
attention.  As  was  stated  in  the  report  from 
the  DoD  Advisory  Panel  on  Streamlining 
and  Codifying  Acquisition  Law,  there  is, 
"widespread  public  pierceptions  that  the  term 
’Government  procurement’  is  synonymous 
with  ’scandal’."  (17)  We  must  be  willing  to 
spend  the  comparatively  few  dollars  that 
education  and  training  will  cost  us,  so  that 
we  may  be  better  stewards  of  taxpayer 
dollars. 

The  Merit  Systems  Protection  Board’s  1992 
study,  warned  that  the  training  area  is  still 
lacking,  especially  Government-provided 
training  courses  which  focuses  on 
procurement.  Many  personnel  have 
completed  basic  courses  (81%  completed 
basic  procurement  management  and  70% 
completed  contract  administration). 
Although  these  statistics  give  us  pause  to 
assess  the  effectiveness  of  training 
programs,  the  study  also  found  that  only 
51%  had  completed  advanced  procurement 
management  and  only  38%  had  finished  the 
advanced  contract  administration  course. 
(20)  This  finding  does  not  show  much 
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improvement  over  the  information  given  in 
the  earlier  paper.  The  lack  of  advance 
training  by  a  large  percentage  of  the 
workforce  is  disturbing  for  several  reasons. 
One  reason  is  that  training  results  in 
employees  being  better  prepared  to  handle 
the  complex  problems  which  arise  in 
contracting.  The  Merit  Systems  Protection 
Board  demonstrated,  statistically,  that  there 
is  a  positive  relationship  between  the 
number  of  training  courses  completed  and 
performance  ratings.  This  was  the  only  area 
where  the  Board  found  a  strong  correlation. 
(20)  A  second  reason  is  that,  with 
personnel  staying  in  the  procurement  field 
for  longer  periods,  there  should  be  a  higher 
percentage  of  personnel  who  have  qualified 
for,  and  completed,  advance  training 
courses.  Just  working  in  the  procurement 
career  field  is  not  enough,  advance  training 
courses  are  required  in  order  for  personnel 
to  be  prepared  to  handle  the  more  complex 
work  which  should  be  assigned  to  the  most 
experienced  workers. 

A  higher  percentage  of  degreed  personnel 
entering  the  field  and  a  slightly  more  stable 
workforce  are  helping  to  develop  a 
professional  acquisition  workforce  for  the 
Government.  But  much  needs  to  be  done, 
particularly  in  the  areas  of  retention  and 
advanced  training.  However,  given  the 
current  budget-cutting  atmosphere,  there 
should  be  concerns  that  promotion  and  wage 
constraints  will  erode  advances  in  retention 
and  that  training  will  be  more  difficult  to 
justify  from  a  strictly  budgetary  viewpoint. 

The  Department  of  Defense  (DoD)  has  made 
strides  towards  a  professional  acquisition 
workforce.  In  1990  Congress  adopted  the 
Defense  Acquisition  Workforce 
Improvement  Act  (DAWIA).  This  Act  is 
intended  to  improve  the  quality  and 
professionalism  of  the  DoD  acquisition 
workforce.  In  addition  to  recognizing 


acquisition  as  a  professional  career  field, 
capped  by  admission  into  a  professional 
Acquisition  Corps,  several  other  policies  are 
established:  requirement  to  improve  the 
education,  training,  and  experience  levels  of 
the  workforce;  a  career  development 
program  for  acquisition  professionals;  a 
career  management  structure  in  DoD;  a 
Defense  Acquisition  University  structure; 
and,  programs  to  assist  acquisition  personnel 
in  their  professional  development.  The  law 
is  being  phased  in  over  a  three  year  period, 
ending  in  October,  1993.  Each  of  the 
Services  is  working  to  implement  the 
requirements  of  the  Act. 

The  Defense  Acquisition  Workforce 
Improvement  Act  has  mandated  certain 
education  requirements.  Those  personnel 
who  hold  degrees  in  other  than  business- 
related  disciplines  must  complete  at  least  12 
semester  hours  of  business-related  courses  to 
enter  a  DoD  Acquisition  Corps.  The 
requirement  for  Contracting  Officer 
positions  is  even  more  stringent;  at  least  24 
semester  hours  of  business-related  courses, 
if  the  individual’s  degree  is  in  a  discipline 
other  than  business-related.  (4) 

There  is  one  major  problem  which  effects 
all  of  DoD  and  effective  implementation  of 
the  Act.  This  problem  is  the  huge  backlog 
of  training  requirements.  Acquisition 
personnel  are  attempting  to  sign  up  for 
required  courses  and  for  newly  created 
courses.  Personnel  needed  for  critical 
acquisition  positions  are  being  given  priority 
to  attend  needed  training.  There  is  a 
concern  that  senior  personnel  are  being  sent 
for  training  irrespective  of  whether  their 
prior  work  experience  made  such  training 
superfluous.  DoD  must  solve  this  problem 
so  that  less  senior  personnel  can  enroll  in 
training  courses  and  become  prepared  to 
take  over  critical  positions.  One  potential 
solution  has  been  preempted  on  an  ad  hoc 
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basis:  although  DAWIA  authorizes  waivers 
of  the  training  requirement,  there  is  great 
reluctance  to  use  waivers,  as  neither 
commands  nor  individuals  want  to  have 
waivers  of  training  requirements  as  part  of 
official  records.  The  problem  of  meeting 
the  training  requirements  is  being  addressed 
by  additional  course  offerings,  and  in  time, 
it  will  be  overcome. 

The  Air  Force,  seemingly  a  leader  in  the 
strive  to  develop  a  professional  acquisition 
workforce,  has  formulated  the  policies 
necessary  for  reaching  the  goal.  The  Air 
Force’s  24  January  1992  Memo,  entitled, 
"Certification  in  the  Contracting  Acquisition 
Professional  Development  Program  (APDP) 
-  ACTION  MEMORANDUM,"  states,  "The 
Acquisition  Professional  Development 
Program  is  designed  to  provide  the 
contracting  community  with  a  structure  that 
ensures  our  people  get  the  necessary 
training,  education,  and  experience  to 
effectively  progress  into  more  responsible 
and  demanding  positions."  (2) 

The  Army  also  appears  confident  in 
reaching  the  goal  of  professionalism.  The 
Army  Research,  Development  and 
Acquisition  Bulletin,  in  describing  the  Army 
Acquisition  Corps  (A AC)  stated,  "The 
program  will  build  a  world  class  acquisition 
workforce."  (1)  The  A  AC  has  been  set  up 
to  provide  civilian  Corps  members  with 
opportunities  for  professional  development 
throughout  their  careers.  For  the  military 
members  of  the  AAC,  the  Military 
Acquisition  Management  Branch  was  formed 
with  career  management  responsibilities.  (3) 

Civilian  agencies  are  much  further  behind 
the  military  in  the  formulation  of  a 
professional  workforce.  The  agencies  are 
working  to  address  the  requirements  of  the 
Office  of  Federal  Procurement  Policy’s 
Letter.  The  information  on  the  program  is 


now  being  disseminated  throughout  agencies 
and  progress  is  being  made  to  implement  the 
requirements.  For  example,  the  Treasury 
Department  has  implemented,  in  June,  1992, 
the  Procurement  Career  Management 
Program  guidelines  which  was  designed  to, 
"Meet  the  Department’s  needs  for 
professional  contracting  and  purchasing 
personnel  and  to  provide,  on  a  planned 
systematic  basis,  highly  qualified  candidates 
for  senior  contracting  and  procurement 
positions  as  they  become  vacant."  (13)  The 
guidelines  recognize  and  incorporate  the  FAI 
Contract  Specialist  Workbook  into  the 
Department’s  program. 

INDUSTRY 

The  second  component  of  the  triumvirate  is 
Industry.  The  replication  of  the  newspaper 
advertisement  analysis  that  was  conducted 
for  the  original  study  showed  that  Industry 
has  made  some  significant  advances.  A 
study  of  167  advertisements  for  contracting 
professionals  appearing  in  the  Washington 
Post  during  the  period  from  26  July  1992  to 
1  February  1993  showed  a  stronger 
commitment  to  professional  requirements. 
The  current  study  found  that  64.44%  of  the 
advertisements  had  degree  requirements,  a 
considerable  improvement  over  the  earlier 
study  which  found  only  20.44%  with  such 
requirements.  One  advertisement  in  the 
current  study  even  gave  the  requirement  of 
a  Bachelor’s  Degree  and  formal  training  in 
acquisition.  However,  this  still  means  that 
over  one-third  of  the  advertisements  did  not 
levy  degree  requirements. 

A  particularly  disturbing  finding  was  that 
only  one  advertisement  stated  a  requirement 
that  the  candidate  be  a  Certified  Professional 
Contracts  Manager  (CPCM),  although  five 
other  advertisements  did  note  that  CPCM 
status  was  desirable.  This  3.46%  mention 
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of  professional  certification  is  an  increase 
from  the  previous  study’s  2.89%,  but  is 
certainly  not  the  increase  that  would  show  a 
true  commitment  to  professionalism. 
According  to  Dr.  David  Lamm,  Professor 
and  Academic  Associate,  Acquisition  and 
Contract  Management,  Naval  Postgraduate 
School,  "The  CPCM  designation  continues 
to  represent  the  highest  level  of  qualification 
in  the  contracting  profession."  (8)  Given 
this  high  regard  it  is  somewhat  surprising 
that  there  is  not  more  recognition  of  the 
certification. 

When  these  first  findings  are  coupled  with 
other  statistics  from  the  advertisements  the 
commitment  to  professionalism  is  really 
questioned.  In  the  current  statistics  only 
2.39%  of  the  advertisements  stated  a 
requirement  for  acquisition  training.  While 
that  is  a  "tremendous  increase"  over  the 
previous  finding  of  0.44%  it’s  still 
negligible.  Compare  the  training 
requirements  to  the  almost  four  percent 
(3.46%)  of  the  advertisements  in  the  current 
study  that  gave  typing  (from  45  words  per 
minute  up  to  65  words  per  minute)  and 
shorthand  as  requirements.  That  percentage 
is  the  same  as  the  number  requesting 
professional  certification  and  significantly 
more  than  acquisition  training.  One 
wonders  what  portion  of  the  contracting 
work  is  seen  as  the  most  important, 
negotiating  appropriate  and  solid  contracts 
or  the  fast  typing  of  memoranda  of  actions. 
Perhaps  a  little  more  understandably, 
computer  skills  are  also  deemed  to  be  very 
important.  Seventy-three  percent  of  the 
advertisements  were  for  positions  that 
required  computer  skills,  with  LOTUS  and 
WORDPERFECT  the  most  often  cited.  If 
frequency  is  an  indicator  of  the  importance 
to  the  company,  then  computer  skills  are 
more  important  than  actual  contracting 
experience,  which  was  mentioned  in  only 
68.26%  of  advertisements. 


Although  great  strides  are  hard  to 
determine,  there  are  signs  that  industry  is 
aware  of  the  need  for  a  professional 
acquisition  workforce,  at  least  on  an 
individual  corporate  basis.  One  example  of 
a  firm  taking  the  responsibility  seriously  is 
Honeywell’s  Space  System  Group  in 
Clearwater,  Florida  which  convinced  Florida 
Institute  of  Technology  to  open  a  satellite 
campus  close  to  their  worksite.  The  success 
of  Honeywell’s  focused  approach  is  proven 
when  it  is  shown  that  50%  of  Space  Systems 
Group’s  contract  management  professionals 
have  advance  degrees  in  related  fields.  In 
addition,  "Honeywell  requires  its  contract 
managers  to  have  a  CPCM  certificate  before 
they  can  be  promoted  to  the  level  of  senior 
contract  representative."  (16)  This  is  not  a 
burdensome  requirement  as  65%  of  their 
contract  managers  are  already  CPCM’S. 
This  is  the  sort  of  dedication  to  training, 
education,  and  professionalism  that  we 
should  see  from  all  corporations. 

A  survey  of  the  advertisements  in  Contract 
Management  found  a  stronger  link  to 
professionalism,  as  would  be  expected. 
Although  there  were  only  seven 
advertisements  from  the  August,  1992  issue 
to  the  February,  1993,  issue,  all  of  the 
advertisements  had  degree  requirements  and 
42.86%  of  the  advertisements  mentioned 
CPCM  as  desirable.  We  expected  this 
percentage  would  be  higher,  given  that  the 
magazine  is  published  by  the  National 
Contract  Management  Association,  Sponsor 
of  the  CPCM  certification.  The  statistics  of 
the  current  study  show  some  progress  from 
the  previous  one,  which  found  only  a 
39.13%  of  the  advertisements  requiring 
certification.  However,  due  to  the  limited 
measurement  base,  this  increase  should  be 
viewed  with  caution. 

The  ultimate  goal,  that  all  Industry  positions 
would  require  acquisition  training, 
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professional  degrees  and  professional 
certification,  is  still  in  the  future,  but  is 
becoming  more  of  a  possibility. 

ACADEME 

The  third  member  of  the  triumvirate. 
Academe,  has  made  impressive  strides  from 
the  previous  study.  There  has  been  a 
noteworthy  increase  in  availability  of 
procurement  degrees.  As  was  cited  in  the 
initial  paper,  a  1980  study  by  the  Federal 
Acquisition  Institute  indicated  that  24 
institutions  offered  Master’s  Degrees,  7 
offered  Bachelor’s  Degrees  and  6  offered 
Associate’s  Degrees.  (11)  By  December, 
1990,  the  number  of  institutions  offering 
degrees  in  contract  management  related 
fields  had  increased  to  54  Master’s  Degrees, 
27  Bachelor’s  Degrees  and  21  Associate’s 
Degrees,  increases  of  770%,  386%,  and 
350%,  respectively.  In  addition,  two 
institutions.  Southeastern  Institute  of 
Technology  and  Arizona  State  University 
are  offering  Doctorate  level  programs  in  the 
field.  (15)  With  few  exceptions,  the 
programs  are  designated  acquisition, 
contract,  procurement  or  purchasing. 

In  addition  to  the  institutions  that  grant 
degrees  in  the  acquisition  field,  academic 
courses  of  17  colleges  and  universities  have 
been  approved  by  the  Interagency  Academic 
Program  Committee  as  substitutable  for 
federal  training  requirements.  (15) 
Certification  programs  are  offered  by  many 
colleges  and  universities,  such  as  the 
University  of  Southern  California  and  the 
University  of  Virginia.  Some  such  as 
Florida  Institute  of  Technology  and  George 
Washington  University  offer  both  degree 
and  certification  programs. 

The  significant  increase  in  college  and 
university  curricula  in  acquisition  shows  the 


greatest  commitment  to  a  professional 
workforce  seen  so  far.  It  was  surprising, 
however,  that  the  synergism,  that  we  need 
and  expect,  did  not  go  just  a  little  bit 
further.  A  review  of  the  last  six  issues  of 
Contract  Management  found  that  only  two 
universities,  George  Washington  University 
and  Florida  Institute  of  Technology, 
advertise  in  the  magazine.  We  would  have 
expected  that  the  increase  in  acquisition 
curriculum  would  be  shown  in  this 
professional  publication. 

Although  not  from  a  college  or  university, 
the  February,  1993,  issue  of  Contract 
Management  did  include  an  advertisement 
from  the  Federal  Publications,  Incorporated, 
listing  contract  training  courses  available 
from  the  firm.  However,  the  advertisement 
stated  only  a  listing  of  courses  and  did  not 
state  whether  a  certificate  is  available  for 
completion  of  courses. 

Government  and  Industry  will  have  to  be  the 
impetus  behind  further  increases  in  degree 
programs  in  acquisition.  The  demand  for  a 
professionally  educated  workforce  will  result 
in  more  degree  programs.  Competition  for 
jobs  in  the  acquisition  corps  will  most  likely 
result  in  individuals  getting  degrees  in  the 
acquisition  and  contract  discipline. 

Some  amount  of  synergism  has  been  noted, 
for  example,  Florida  Institute  of 
Technology,  in  making  acquisition  courses 
easily  available  to  acquisition  personnel,  has 
located  offices  and  classrooms  near  the  work 
sites  of  acquisition  offices  such  as  the  Army 
Material  Command,  NASA,  and  the  Army 
Lx)gistics  Management  College.  The 
Government  recognized  the  importance  of 
teamwork  and  required,  in  OFPP’s  Policy 
Letter  92-3,  that  the  Director  of  FAI, 
"Assist  colleges  and  universities  in 
establishing  procurement  and  acquisition 
courses  as  part  of  continuing  education. 
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associate,  baccalaureate,  and  graduate 
programs."  (14)  This  synergism  includes 
Industry,  as  was  noted  earlier,  with  Florida 
Institute  of  Technology’s  work  with 
Honeywell. 

SUMMARY 

Although  strides  have  been  made,  some 
great,  since  looking  at  this  subject  in  1983, 
much,  much  more  still  needs  to  be 
accomplished.  Government,  Industry  and 
Academe  have  made  some  combined  efforts 
toward  the  goal.  The  Defense  Acquisition 
Workforce  Improvement  Act  and  the  Office 
of  Federal  Procurement  Policy’s  Policy 
Letter  92-3  (Procurement  Professionalism 
Program  Policy  -  Training  for  Contracting 
Personnel)  have  levied  professional 
requirements  for  the  Government’s 
acquisition  personnel.  Industry  is,  more 
than  ever  before,  requiring  professional 
credentials  of  job  applicants.  There  has 
been  a  great  increase  in  the  number  of 
degree  programs  in  acquisitions  and 
contracts  offered  at  our  colleges  and 
universities. 

It  is  hoped  that  the  goal  of  establishing  a 
truly  professional  acquisition  workforce  can 
be  reached  when  we  review  the  status  of 
acquisition  professionalism  in  2003. 
However,  this  is  unlikely  without  an  all  out 
and  concerted  effort  by  Government, 
Industry  and  Academe. 
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ABSTRACT 

This  paper  is  the  report  of  a  continuing 
research  effort  to  develop  and  apply  a  method  for 
classifying  various  goods  procured  by  the  Federal 
Government  according  to  key  characteristics  and 
attributes  exhibited  by  these  goods.  The  primary 
objective  has  been  to  develop  a  classification 
scheme  (model)  that  can  be  used  by  practitioners 
and  researchers  in  grouping  goods  along  a 
continuum  from  simple  to  complex  utilizing 
strategic  characteristics  essential  to  the  buying 
process. 

A  taxonomical  model  for  classifying  goods 
versus  characteristics  was  developed  utilizing 
cluster  analysis  (1  j  and  was  subsequently 
validated  through  continuing  data  collection  and 
analysis.  (2]  The  latter  stages  of  the  research 
have  focused  on  clarifying  application  of  the 
model  in  the  buying  environment  including 
refinement  of  acquisition  policies  and  procedures 
reflecting  differences  between  classes  of  goods, 
use  of  market  research  techniques,  training  and 
educating  the  workforce,  developing  a  better 
understanding  of  the  relationships  between 
goods,  and  developing  staffing  models,  to  name 
a  few. 

INTRODUCTION 

One  of  the  most  potentially  significant 
concepts  proposed  in  the  field  of  Government 
contracting  is  the  idea  that  contracting  is  a 
science.  This  came  about  as  a  result  of  the 
growing  complexity  and  increasing  difficulties  in 
the  procurement  process.  These  problems 
emphasized  the  need  for  procurement  research 
as  a  means  of  obtaining  a  better  understanding  of 
the  intricacies  and  effects  of  the  process.  The 
key  to  effective  research  is  the  use  of  a 
systematic  approach  to  solving  problems.  The 
concept  of  contracting  as  a  science  implies  that 
a  systematic  and  organized  process  can  be 


employed  for  development  and  validation  of 
contracting  knowledge.  [3] 

Among  the  requirements  identified  as 
differentiating  sciences  from  other  disciplines  is 
the  description  and  classific^ion  of  the  subject 
matter.  [4]  The  pursuit  of  this  requirement  has 
lead  to  the  development  of  a  taxonomical 
structure  for  classifying  goods  procured  by  the 
Federal  Government.  (See  Figure  1  for  the 
Taxonomy  as  developed  to  date.) 

The  major  benefits  proposed  are  that  accurate 
questions  could  be  asked  as  to  how  the 
perceived  order  of  goods  has  arisen  and  how 
best  to  maintain  or  improve  it,  the  structure 
necessary  for  identifying  the  types  of  goods 
purchased  by  the  Government  in  a  profile  that 
lends  itself  to  increased  visibility,  and  accurate 
determination  of  the  most  viable  procurement 
strategies  for  buying  certain  products. 

At  this  stage  of  the  taxonomy’s  development, 
it  is  appropriate  to  consider  its  usefulness  by 
examining  practical  applications  and  benefits.  This 
is  necessary  for  two  reasons.  First,  it  is  important 
to  bridge  the  gap  between  researchers  and 
practitioners.  “Many  people  in  the  scientific 
community  believe  that  research,  like  virtue,  is  its 
own  reward.  Procurement  research  is  not  quit 
that  exalted.  Harried  administrators  and  impatient 
Congressmen  want  to  see  results-practical 
applications  that  will  improve  the  procurement 
process."  [5] 

The  second  reason  to  consider  practical 
applications  is  to  provide  direction  for  future 
research  and  refinement  of  the  taxonomical 
model.  The  most  important  evaluation  criterion  of 
a  classification  scheme  is  how  useful  it  is  in 
helping  solve  problems.  [4]  In  order  to  refine  and 
advance  the  taxonomy,  the  applications  for  which 
the  model  is  intended,  and  the  resulting  benefits, 
should  be  understood. 
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CLASSIFICATION  STUDIES 

Classification  studies  conducted  in  the  field  of 
Government  procurement  have  been  a 
continuation  of  the  concept  of  contracting  as  a 
science.  This  concept  was  first  proposed  by 
Robert  Williams  and  Paul  Arvis  in  a  paper 
presented  to  the  Federal  Acquisition  Research 
Symposium  in  1985  [6],  and  subsequently 
examined  in  research  conducted  by  Steven  Park 
in  1986.  [3]  The  concept  of  framing  the  field  of 
contracting  as  a  science  is  useful  because  it 
advocates  a  systematic  and  organi2ed  method  for 
exploring  and  defining  the  discipline. 

Government  procurement  is  a  complex  field 
with  conflicting  requirements  placed  on  the 
process  by  the  many  goals  of  the  system.  For 
example,  the  Federal  procurement  process  is 
expected  to  obtain  quality  products  at  fair  and 
reasonable  prices.  At  the  same  time,  the  process 
serves  as  an  instrument  of  foreign  policy  through 
the  foreign  military  sales  (FMS)  program,  social 
policy  through  socioeconomic  programs,  and 
fiscal  policy  through  fluctuations  in  spending 
levels.  The  procurement  process  serves  as  a  tool 
for  carrying  out  many  other  policy  goals.  The 
result  is  a  complex  process  with  requirements 
often  at  odds  with  one  another.  Government 
procurement  stands  to  benefit  from  an  organized 
and  systematic  approach  to  studying  the  field. 

TAXONOMY  OF  GOODS 

Origin 

The  taxonomical  model  developed  during  this 
study  has  its  roots  in  the  field  of  marketing  and, 
in  particular,  the  classification  scheme  developed 
by  Gordon  Miracle.  [7]  The  conceptual  basis  for 
the  classification  scheme  was  to  classify 
Government  goods  in  a  way  that  offers  strategic 
insight  into  the  buying  process.  The  goal  was  to 
create  a  classification  scheme  that  would  highlight 
the  various  categories  of  goods  and  their  related 
characteristics  to  allow  streamlining  and  tailoring 
of  contracting  policies,  methodologies,  and 
procedures. 

Ojjectives 

There  are  several  basic  objectives  of 
classification  in  any  discipline.  Classification  is 
rarely  viewed  as  an  end  in  and  of  itself. 
Classification  systems  are  generally  viewed  as  a 
means  to  improve  the  abiiity  to  interpret,  predict, 
or  control  some  facet  of  performance.  [8] 

The  primary  objective  of  classification  is  to 


describe  the  structure  and  relationships  of 
constituent  objects  with  regard  to  each  other  and 
similar  objects.  Classification  simplifies  these 
relationships  in  such  a  way  that  general 
statements  can  be  made  about  classes  of  objects 
and  boundaries  between  objects  can  be  identified 
through  differentiation.  It  is  easy  to  observe 
structure  when  it  is  obvious  and  discontinuous. 
For  example,  horseshoe  crabs  are  a  unique 
species  very  different  from  their  nearest  relatives. 
Classification  is  most  useful  when  the  structure  is 
less  obvious.  [9] 

Another  objective  of  classification  is  to 
achieve  economy  of  memory  and  facilitate 
communication.  Without  the  ability  to  summarize 
information  and  attach  a  label  to  it, 
communication  would  be  impossible.  [8]  Another 
objective  of  classification  is  ease  of  manipulation 
of  information.  Classification  achieves  ease  of 
manipulation  because  the  system  consists  of  taxa 
that  can  be  easily  named  and  related  to  each 
other.  If  relationships  are  complex,  labeling  or 
handiing  of  the  taxa  will  not  be  easy.  [9] 

Usefulness 

In  generic  terms,  the  application  of  a 
taxonomy  to  a  set  of  facts  or  objects  results  in 
adding  more  information  to  those  facts  or  objects 
by  revealing  patterns,  enabling  predictions,  and 
guiding  various  decisions.  [10]  Classification  can 
uncover  important  differences  that  may  not 
otherwise  be  evident,  or  expose  weaknesses  in  an 
existing  knowledge  base  that  require  further 
investigation. 

>n  describing  the  liselulness  of  a  human 
performance  taxonomy,  Fleishman  and 
Quaintance  suggest  that  the  taxonomy  has 
important  practical  and  scientific  implications  in  a 
variety  of  fields.  They  further  provide  that  a 
number  of  ostensibly  disparate  problems  can  be 
drawn  together  and  viewed  in  a  new  light  by  the 
application  of  a  taxonomy.  They  divide 
taxonomical  uses  into  scientific-theoretical  and 
applied-practical.  The  scientific-theoretical 
applications  can  be  thought  of  as  generic  uses. 
That  is,  they  can  be  useful  in  any  discipline. 
Among  the  most  important  areas  of  usefulness 
are  the  following:  (1)  conducting  literature 
reviews,  (2)  establishing  better  bases  for 
conducting  and  reporting  research  studies  to 
facilitate  their  comparison,  (3)  standardizing 
methods  of  study,  (4)  generalizing  research  to 
new  tasks,  (5)  exposing  gaps  in  knowledge,  and 
(6)  assisting  in  theory  development.  [8] 
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Taxonomies  and  Contracting 

Taxonomies  can  achieve  the  same  basic 
objectives  in  contracting  as  suggested  above. 
The  taxonomy  of  goods  procured  by  the  Federal 
Government  could  serve  the  overall  role  of 
advancing  procurement  research,  which  would  in 
turn  improve  the  procurement  system. 

Since  the  taxonomy  is  based  on  the 
characteristics  of  goods  deemed  important  in  the 
buying  process,  it  could  provide  insight  into  the 
structure  and  relationships  of  goods  procured  by 
the  Federal  Government  from  the  perspective  of 
the  buying  process.  The  taxonomy  could 
enhance  observation  and  clarify  relationships 
among  the  goods.  From  this,  general  statements 
could  be  made  about  individual  categories  of 
goods  which  could  help  guide  decisions  in  the 
procurement  process.  The  differences  identified 
between  the  goods  in  different  categories,  as  well 
as  the  similarities  between  goods  within  each 
category,  could  provide  valuable  insight  for 
decision  making.  And  as  suggested  earlier,  this 
process  may  be  most  useful  when  considering 
goods  with  less  discernable  differences  than 
those  between  a  five  cent  pencil  and  a  billion 
dollar  ship. 

By  describing  relationships  among  goods,  the 
taxonomy  could  result  in  hypotheses  about  the 
relationships.  In  turn,  these  hypotheses  could  be 
tested,  providing  explanations  for  the  relationships 
and  expanding  the  knowledge  base. 

The  taxonomy  of  goods  could  also  achieve 
economy  of  memory.  This  is  not  to  suggest  that 
buyers  have  a  need  to  memorize  the 
characteristics  of  goods  in  each  category,  but 
rather,  a  taxonomy  could  facilitate  communication 
because  it  enhances  the  understanding  of  the 
goods  and  streamlines  the  communication 
process.  Each  specific  characteristic  of  a  buy 
need  not  be  spelied  out  entirely,  the  category 
label  could  summarize  information  instead. 

A  taxonomy  of  goods  could  also  facilitate 
ease  of  manipulation  and  retrieval  of  information 
in  contracting.  This  holds  true  from  both  a 
theoretical  research  standpoint  and  a  practical 
application  standpoint.  By  classifying  and 
labeling  categories  of  goods,  the  taxonomy 
provides  a  framework  for  manipulation  and 
retrieval  of  contracting  information.  From  the 
practical  application  perspective,  this  provides  an 
efficient  means  for  operationalizing  the  guidelines 
developed  from  the  taxonomy.  From  the 
perspective  of  procurement  research,  it  facilitates 
organized  and  systematic  study. 


Finally,  the  taxonomy  of  goods  procured  by 
the  Federal  Government  could  achieve  scientific- 
theoretical  usefulness.  In  much  the  same  manner 
described  in  the  previous  section,  the  taxonomy 
could  serve  procurement  research  by  enhancing 
literature  reviews,  establishing  a  better  base  for 
conducting  and  reporting  research  studies  to 
facilitate  their  comparison,  standardizing  methods 
of  study,  generalizing  research  results  to  new 
areas,  exposing  gaps  in  knowledge,  and  assisting 
in  theory  development.  All  of  these  would  expand 
the  body  of  knowledge  in  contracting. 

Potential  Applications 

Based  on  the  discussion  presented  thus  far, 
potential  areas  for  application  of  the  taxonomy  of 
goods  procured  by  the  Federal  Government  are 
listed  below.  Although  not  exhaustive,  this  list 
does  present  the  most  fertile  areas  in  applying  the 
taxonomy. 

1.  Market  Research 

2.  Policy  Guidance 

3.  Training/Education 

4.  Staffing 

5.  Procurement  Reviews 

6.  Budgeting 

7.  Legislative  Development 

8.  Regulations/Procedures 

9.  Contract  Type  Selection 

10.  Contracting  Method 

1 1 .  Change  Control 

12.  Break-out  Decisions 

13.  Unsolicited  Proposal  Procedures 

14.  Industrial  Base  Decisions 

15.  Profit  Guidelines 

16.  Source  Selection  Procedures 

17.  Administrative  Procedures 

18.  Clause  Selection 

19.  Specification  Selection 

20.  Configuration  Control 

21.  Independent  Research  and  Development 
(IR&D)  Policy 

22.  Acquisition  Strategy 

23.  Workload  Management 


APPLICATIONS 
Market  Research 

Introduction 

Perhaps  one  of  the  most  beneficial 
applications  of  the  taxonomical  structure  for 


411 


classifying  goods  procured  by  the  Federal 
Government  is  in  the  area  of  market  research. 
Market  research  is  an  element  of  the  acquisition 
planning  process  which  has  been  overlooked  in 
Government  procurement.  [11]  Market  research 
is  an  area  where  the  taxonomical  methods  used 
in  the  discipline  of  marketing  can  be  most  directly 
extrapolated  to  the  contracting  field.  This 
application  of  the  taxonomy  could  help  shore  up 
a  weakness  in  Government  procurement, 
particularly  in  these  times  of  declining  budgets 
and  a  shrinking  industrial  base. 

Market  research  in  the  field  of  contracting  can 
be  viewed  from  two  perspectives.  From  a  macro 
view,  market  research  can  advance  the  body  of 
knowledge  in  contracting  and  improve  the 
procurement  process  by  revealing  trends  in  the 
market  that  need  to  be  reversed,  or  practices  that 
work  better  than  others,  or  policies  that  enhance 
competition  in  general.  Viewed  from  a  micro 
perspective,  market  research  can  be  used  to 
enhance  competition  or  gain  an  understanding  of 
production  processes  for  a  specific  procurement. 
Market  research  in  contracting  is  concerned  with 
an  examination  of  the  producers  or  suppliers  of 
goods  and  services  in  order  to  better  serve  the 
customers  of  the  procurement  process. 

Market  Research  in  Contracting 

Market  research  became  a  statutory 
requirement  with  the  passage  of  the  Competition 
in  Contracting  Act  (CICA)  of  1984.  The  Act 
specifically  states:  "In  p'^mning  for  the 
procurement  of  property  or  services,  an  executive 
agency  shall  use  advance  procurement  planning 
and  market  research..  ."  [12]  Market  research  is 
intended  to  offer  an  opportunity  to  reduce  barriers 
to  competition  by  improving  the  information 
availaL.e  to  the  contracting  officer  during  the 
acquisition  planning  phase.  The  statute  makes 
market  research  the  central  function  in  both 
achieving  competition  where  possible  and 
justifying  the  circumstances  when  it  is  not.  [13] 

In  a  recent  study,  a  definition  of  market 
research  was  proposed  as,  "the  collection  and 
analysis  of  data  to  improve  the  quality  of  specific 
decisions  which  must  be  made  within  the  existing 
framework  of  the  procurement  process."  [12] 
This  definition  recognizes  that  there  are  many 
aspects  of  the  procurement  process  that  stand  to 
be  improved  through  market  research.The  study 
also  presented  both  a  narrow  view  and  a  broad 
view  of  market  research  pertaining  to  the  Federal 
procurement  process,  and  advocates  adoption  of 


the  broad  view.  [12]  "One,  the  narrow  view, 
holds  that  the  purpose  of  market  research  is 
merely  to  Identify  potential  sources  of  supply.  The 
other,  the  broad  view,  holds  that  market  research 
involves  far  more  than  identification  of  potential 
sources  of  supply.  In  fact,  the  broad  view  is  that 
the  requirement  involves  understanding  the 
market  place  and  conducting  the  methodical 
research  that  is  often  times  necessary  to  develop 
that  understanding. 

It  is  clear  that  a  thorough  market  research 
capability  developed  within  the  Federal 
procurement  system  would  substantially 
strengthen  the  ability  of  the  Government  to  use 
and  enhance  the  purchasing  process  more 
effectively,  including,  but  not  limited  to,  generating 
competition  [11] 

Stewart  delineated  five  principal  elements  of 
an  effective  market  research  program  based  on 
an  analysis  of  literature  and  observations  of  both 
Government  and  private  industry  practices:  (1) 
criteria  for  project  selection,  (2)  proper  research 
and  analysis  skills,  (3)  a  methodical  approach,  (4) 
timely  information,  and  (5)  effective 
communication  of  findings.  [12] 

Application  of  the  Taxonomy 
in  Market  Research 

Implementation  of  the  taxonomy  of  goods 
procured  by  the  Federal  Government  would 
facilitate  market  research  in  several  respects.  The 
taxonomy  would  first  be  operationalized,  with  ail 
goods  procured  by  the  Federal  Government 
classified.  Then  the  classification  scheme  could 
serve  as  the  basis  for  organizing  market  research. 
Research  on  the  market  could  be  done  on  a 
category-by-category  basis,  providing  an 
organizational  framework  for  systematic  study. 
Elements  of  the  scientific-theoretical  usefulness  of 
the  taxonomy  described  earlier  would  be  applied 
in  market  research.  If  market  research  were 
conducted  along  the  lines  of  the  categories  of  the 
taxonomy,  literature  reviews  would  be  made 
easier  because  information  on  the  market  could 
be  accessed  by  the  category  to  which  an  item 
belongs.  The  categories  would  serve  as  the 
bases  for  conducting  and  reporting  research 
studies  to  facilitate  their  comparison 
Observations  of  goods  within  individual  categories 
may  be  generalized  to  other  goods  in  the  same 
categories.  For  example,  suppose  that  personal 
computers  and  ship  positioning  computers  were 
both  identified  to  the  advanced  category  of 
goods.  Perhaps  goods  in  this  category  would  be 
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characterized  by  a  high  degree  of  maintenance. 
Observations  of  the  successful  warranty  terms  or 
maintenance  contracts  used  in  the  more 
commonly  procured  personal  computers  could  be 
generalized  and  used  when  buying  the  less 
frequently  purchased  positioning  computers. 

Areas  where  greater  research  is  needed  may 
be  exposed.  At  the  present  time,  there  is  no 
framework  consistently  used  throughout  Federal 
procurement  for  the  accumulation  and  storage  of 
market  research  data.  Adoption  of  the  taxonomy 
could  provide  a  consistent  approach  for 
accumulation  of  information  on  the  market. 

As  discussed  earlier,  Stewart  identified  five 
principal  elements  of  an  effec.iive  market  research 
program  The  taxonomy  would  facilitate  several 
of  these  elements.  The  first  element,  criteria  for 
project  selection,  would  be  enhanced  by  the 
taxonomy  because  one  of  the  criteria  for  project 
selection  should  be  simply  whether  or  not  there  is 
any  existing  information  available  on  the  project 
at  hand  For  example,  a  buyer  has  a  requirement 
for  a  pneumatic  valve.  Before  initiating  a  market 
research  project,  the  buyer  could  access  a  data 
base  by  the  category  of  the  valve,  perhaps 
moderate,  to  see  if  there  is  any  market  research 
information  already  available.  If  so,  the 
information  could  be  used,  and  no  new  project 
would  be  required.  If  not,  a  market  research 
project  may  be  initiated.  The  taxonomy  would 
provide  a  means  for  searching  for.  and  utilizing, 
existing  information,  or  confirming  that  none 
exists. 

Another  element  of  an  effective  market 
research  program  is  a  methodical  approach  which 
the  taxonomy  could  provide.  Market  research 
could  begin  with  the  simple  or  complex  category, 
or  any  category  between  the  two.  Research 
could  be  conducted  on  that  category  on  a 
consistent,  organized  and  continuous  basis, 
methodically  building  the  knowledge  base. 
Framing  market  research  in  this  manner  would 
prevent  duplicating  efforts  or  missing  important 
information. 

Market  research  information  would  be 
provided  in  a  more  timely  fashion  because  the 
taxonomy  would  allow  cataloging  of  the 
information.  If  a  buyer  needs  information  on  a 
moderate  good,  perhaps  a  galley  oven, 
information  could  be  quickly  retrieved  by 
reviewing  the  data  base  accumulated  under  the 
heading  of  the  moderate  category  This  would 
allow  quick  information  retrieval,  and  prevent 
having  to  start  at  the  beginning  with  each  market 


research  project. 

Finally,  the  effective  communication  of 
findings  would  be  simplified  by  the  taxonomy. 
Again,  this  is  a  function  of  indexing  market 
research  along  the  categories  of  goods.  This 
would  provide  those  who  need  the  information  an 
effective  access  mechanism  to  the  information. 
Organizing  market  research  around  the  taxonomy 
would  allow  findings  to  be  related  back  to  the 
body  of  knowledge  in  the  same  manner, 
encouraging  more  effective  communication  of 
findings. 

From  the  macro  perspective,  market  research 
framed  by  the  taxonomy  may  reveal  certain 
categories  of  goods  that  are  particularly  well- 
suited  for  procurement  from  small  business, 
which  should  then  be  targeted  as  such.  On  the 
other  hand,  market  research  may  reveal  a 
category  of  goods  where  there  is  little  small 
business  participation.  Market  research  could 
then  be  initiated  to  identify  and  remove  the 
barriers  to  small  business  participation.  Principles 
discovered  in  studying  categories  where  small 
business  participation  is  exemplary  could  be 
emulated  in  areas  where  there  has  been  little 
success.  For  example,  suppose  market  research 
reveals  that  there  is  a  large  number  of  small 
businesses  succeeding  in  the  advanced  category 
of  goods,  but  few  in  the  less  complex  moderate 
category.  It  may  seem  unusual  that  small 
businesses  are  succeeding  in  the  more  complex 
category  rather  than  the  simpler  category.  Study 
could  then  be  conducted  to  determine  the 
reasons  for  the  difference.  Perhaps  the  difference 
is  that  in  the  advanced  category,  goods  are 
characterized  by  a  higher  degree  of  technical 
complexity  than  the  moderate  category,  but  less 
capital  investment  is  required.  It  may  be 
discovered  that  small  businesses  are  very  skilled 
technically,  but  they  do  not  have  access  to  capital 
for  the  equipment  required  for  goods  in  the 
moderate  category.  A  concerted  effort  could  then 
be  made  to  provide  access  to  capital  for  small 
businesses  in  order  to  succeed  in  the  moderate 
category  of  goods. 

Organizing  market  research  in  relation  to  the 
taxonomy  may  force  recognition  of  characteristics 
shared  by  goods  from  seemingly  different 
industries  for  comparative  purposes.  From  the 
perspective  of  an  individual  procurement,  or  micro 
view,  the  taxonomy  could  provide  access  to 
market  research  information  based  on  the 
category  of  the  item  being  procured  For 
example,  a  particular  procurement  may  appear 
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unique,  with  only  one  contractor  capable  of 
responding  to  the  need.  Since  goods  are 
classified  according  to  inherent  characteristics 
deemed  important  in  the  buying  process,  rather 
than  physical  characteristics,  the  taxonomy  may 
provide  insights  not  otherwise  considered.  Based 
on  the  category  to  which  the  item  being  procured 
belongs,  market  research  centered  around  the 
taxonomy  may  reveal  other  companies  that  are 
capable  of  manufacturing  the  item. 

In  addition  to  enhancing  market  competition, 
market  research  could  also  provide  access  to 
information  required  for  individual  negotiations, 
including  the  state  of  technological  change, 
manufacturing  processes,  warranty  practices,  and 
factors  affecting  prices. 

For  an  organization  buying  bulk  items,  such 
as  fuels,  market  research  may  reveal  lower  priced 
product  substitutes  such  as  another  grade  or  type 
of  fuel.  Market  research  may  provide  quick 
access  to  price  trends  so  that  an  organization  can 
time  their  buys  accordingly,  such  as  making  a 
large  bulk  purchase  instead  of  a  series  of  small 
buys  as  prices  are  increasing.  It  may  identify 
goods  in  markets  which  are  likely  to  be  volatile,  or 
indicate  the  likelihood  of  strong  or  weak 
commercial  demand  for  the  goods,  assisting  in 
procurement  planning. 

In  summary,  implementation  of  the  taxonomy 
of  goods  would  provide  several  benefits  in  the 
area  of  market  research.  In  addition  to  identifying 
sources  of  supply  by  individual  category  to 
enhance  competition,  it  could  also  serve  as  the 
framework  for  conducting  methodical  research, 
and  enhance  understanding  of  the  market.  The 
taxonomy  would  be  a  tool  used  by  the 
procurement  workforce  that  would  make  market 
research  a  viable  process. 

Procurement  Regulation 

Introduction 

Within  the  field  of  Government  contracting, 
Robert  Judson’s  profile  of  the  acquisition 
environment,  serves  as  the  basis  for  promising 
taxonomical  applications.  "Often,  critics  of  the 
acquisition  process  assume  that  the 
characteristics  of  purchasing  ordinary  consumer 
goods  can  be  readily  transferred  to  the 
acquisition  of  unique  systems.  It  is  the  author's 
hope  that  this  profile  will  suggest  restraint,  in 
some  small  way,  in  the  mad  dash  to  legislate  and 
regulate  the  acquisition  of  uncertain  products  as 
if  such  undertakings  were  simple  variations  of  the 


consumer  purchasing  process  and  only  need  to 
be  pressed  by  law  and  regulation  into  that  familiar 
mold."  [14] 

The  obvious  implication  is  that  buying 
commercial,  perhaps  simple,  items  at  one  end  of 
the  spectrum  is  a  different  process  than  buying 
uncertain,  perhaps  complex,  items  at  the  other 
end  of  the  spectrum.  These  differences  should  be 
recognized  not  only  in  procurement  laws  and 
regulations,  but  in  policies  and  procedures  as 
well.  These  separate  areas  of  application  are 
distinct,  however  they  do  share  the  common 
threads  of  shaping  and  guiding  the  procurement 
process,  some  more  rigidly  than  others.  As  such, 
the  logic  underlying  the  usefulness  of  the 
taxonomy  is  similar,  as  are  the  benefits,  in  all  of 
these  areas.  These  areas  will  be  collectively 
referred  to  as  regulation. 

The  Judson  profile  suggests  that  regulations 
are  being  written  for  unique  items,  such  as  major 
weapon  systems,  as  if  they  were  simple 
commercial  items.  This  idea  can  be  taken  a  step 
further.  There  are  laws  and  regulations  in 
existence  that  have  been  written  with  the 
purchase  of  complex  items  in  mind  that  are 
needlessly  applied  to  the  purchase  of  simple 
items,  creating  an  unnecessary  administrative 
burden. 

Application  of  the  Taxonomy 

in  Procurement  Regulation 

The  taxonomy  of  goods  procured  by  the 
Federal  Government  could  prove  to  be  a  very 
useful  tool  in  shaping  procurement  regulation.  If 
all  goods  were  identified  to  their  respective 
categories,  ranging  from  simple  to  complex, 
regulation  could  be  tailored  and  applied  to 
individual  categories.  The  possibilities  of  such  a 
system  are  most  apparent  when  comparing  the 
extreme  categories  of  simple  and  complex  goods. 
However,  refinements  could  also  be  made 
between  the  middle  categories  which,  when 
applied  Government-wide,  would  be  extremely 
beneficial. 

By  viewing  goods  that  the  Government  buys 
in  separate  categories  rather  than  as  one  large 
homogeneous  group,  a  relationship  between  the 
characteristics  of  the  products  within  each 
category  and  appropriate  regulation  could  be 
determined.  Refinements  could  be  made  between 
individual  categories  as  well  as  within  the  extreme 
ends  of  the  simple-to-complex  scale.  For 
example,  at  the  complex  range  of  goods,  such  as 
major  weapon  systems,  two  programs  may  be 
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classified  as  acquisition  category  I  programs 
because  of  the  dollar  value  of  the  programs.  As 
such,  the  two  programs  would  be  subject  to  the 
same  regulation.  The  taxonomy  may  reveal  that 
the  goods  procured  in  one  program  belong  in  the 
advanced  category  rather  than  the  complex 
category,  thereby  allowing  some  relief  or 
additional  streamlining  of  the  acquisition  process. 
A  possible  scenario  for  this  would  be  a  major 
non-developmental  item  (NDI)  program  compared 
to  a  major  research  and  development  program. 
Within  a  single  major  weapon  system,  some  of 
the  items  procured  may  be  complex,  while  the 
taxonomy  may  indicate  that  other  items  within  the 
program  are  more  appropriately  classified  as 
simple,  basic,  or  moderate,  allowing  for  less 
regulatory  control. 

In  these  times  of  declining  budgets  in  the 
defense  industry,  the  Government  must  pay  close 
attention  to  the  defense  industrial  base.  The 
taxonomy  of  goods  procured  by  the  Federal 
Government  could  help  shape  and  implement  a 
comprehensive  industrial  base  policy.  It  could 
highlight  the  category  or  categories  of  goods 
where  there  are  few  sources  of  supply,  perhaps  a 
key  link  to  the  industrial  base.  It  may  show  that 
there  is  a  high  degree  of  homogeneity  in  all  but 
one  of  these  categories,  suggesting  that  concern 
for  a  diminishing  industrial  base  need  only  be 
directed  towards  a  single  category.  Or  perhaps, 
although  there  are  few  producers  in  a  particular 
category,  the  number  of  goods  in  the  category  is 
few,  with  no  anticipation  of  increasing  needs. 
These  circumstances  may  lead  to  adoption  of  a 
policy  that  allows  market  forces  to  take  their 
course. 

Another  possibility  is  that  the  taxonomy 
reveals  categories  characterized  by  different 
degrees  of  customization.  Customization  may  be 
a  key  link  to  the  industrial  base.  Goods  with  little 
customization  seem  likely  to  be  commercial 
goods,  which  translates  to  alternate  markets  for 
companies.  If  there  is  little  customization  in  three 
of  the  categories  of  goods,  it  may  indicate  that 
these  categories  have  strong  commercial  markets, 
meaning  there  is  no  need  for  concern  with  the 
industriai  base  in  these  categories.  A  fourth 
category  of  goods  may  have  a  medium  amount  of 
customization,  but  many  sources  of  supply,  again 
indicating  iittle  need  for  concern  with  industriai 
base  policy  since  there  are  many  sources. 
However,  the  fifth  category  of  goods  may  be 
made  exciusiveiy  for  the  Government.  In  this 
case,  the  Government's  industrial  base  policy  may 


be  to  take  a  hands-off  approach  towards  the  first 
four  categories.  For  the  fifth  category,  the 
Government  may  need  to  take  definite  actions  to 
maintain  an  essentiai  industriai  capabiiity. 

As  a  way  of  assisting  firms  that  make  that  fifth 
category  of  goods,  actions  that  would  assist  the 
industry  could  be  linked  to  that  category  of 
goods,  providing  for  concentrated,  efficient  action. 
For  example,  in  order  to  strengthen  firms  making 
this  category  of  goods,  the  Government  may 
decide  to  liberalize  independent  research  and 
deveiopment  (iR&D)  funding  and  iink  it  to 
commerciai  product  development,  specifically  for 
firms  making  this  category  of  goods,  thereby 
heiping  to  maintain  their  strength. 

An  exampie  of  the  concept  of  iinking  research 
funding  to  commerciai  product  deveiopment  is 
the  Small  Business  innovative  Research  (SBIR) 
program.  Under  iegisiation  being  developed, 
SBIR  program  funding  would  have  a  stronger  link 
to  commercial  product  possibilities.  An  important 
consideration  in  the  evaluation  for  funding  will  be 
based  on  the  applicant’s  pians  to  market  the 
product  commerciaiiy.  [15]  The  taxonomy  wouid 
aiiow  this  concept  to  be  linked  to  the  category  of 
goods  where  it  is  most  needed,  rather  than 
across  the  board,  thereby  maximizing  the  benefits 
of  iimited  resources. 

Another  situation  may  be  that  market  research 
done  on  this  category  of  goods  has  revealed 
impediments  to  business  operations  such  as  profit 
considerations  focused  too  heavily  on  capital 
equipment  investment,  or  excessive  use  of  design 
specifications.  A  concerted  effort  could  be 
directed  towards  manufacturers  of  this  category 
of  goods  to  improve  the  circumstances.  Perhaps 
unique  considerations  must  be  made  in 
determining  profit  objectives,  such  as  shifting 
emphasis  from  capital  equipment  investment  to 
commercialization  potential,  ora  special  emphasis 
on  eliminating  design  specifications,  which  tend  to 
inhibit  innovation  and  create  inefficiency.  Perhaps 
this  category  of  goods  may  be  targeted  for 
multiyear  contracting.  In  essence,  the 
Government  could  develop  and  tailor  policies 
taking  into  account  information  revealed  from 
implementation  of  the  taxonomy. 

Implementation  of  the  taxonomy  of  goods 
could  provide  a  sound  framework  for  selectively 
applying  regulation.  In  many  cases  now, 
regulation  is  either  applied  across  all  procurement 
or  linked  artificially  to  selected  dollar  values.  For 
example,  small  purchase  procedures  are  presently 
limited  to  procurements  under  $25,000.  It  may  be 
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more  logical  to  link  small  purchase  procedures  to 
the  characteristics  of  the  goods  rather  than  dollar 
value.  The  practice  of  allowing  oral  solicitations 
can  be  more  logically  linked  to  the  characteristics 
of  the  goods,  such  as  customization  or  sources  of 
supply.  Oral  solicitations  would  be  called  for 
when  an  item  is  not  customized,  because  there 
would  be  no  need  for  detailed  specifications,  and 
there  would  probably  be  a  commercial  market  for 
the  item.  This  combined  with  many  sources  of 
supply  would  indicate  that  market  forces  set  a  fair 
and  reasonable  price  for  the  item.  Allowing  oral 
solicitations  where  appropriate  based  on  the 
characteristics  of  the  goods  would  save  the  time 
and  cost  of  creating  formal  written  solicitations. 

Small  purchase  procedures  are  basically 
designed  to  streamline  the  procurement  process, 
reduce  administrative  burden,  and  promote 
efficient  and  economical  practices.  Small 
purchase  procedures  balance  the  need  for  control 
with  efficiency  and  low  administrative  costs.  The 
success  of  these  procedures  could  more  logically 
be  linked  to  the  level  of  complexity, 
customization,  documentation,  or  item  attention 
rather  than  the  dollar  value.  The  taxonomy  could 
identify  the  category  or  categories  of  goods  that 
have  the  characteristics  which  lend  themselves  to 
streamlined  procurement  procedures. 

Procurement  Training 

Introduction 

"A  strong  and  viable  training  and  educational 
program  is  fundamental  to  strengthening  the 
DoD’s  acquisition  process."  [16]  Presently, 
training  systems  are  very  fragmented  and 
diffused  Training  and  education  need  to  be 
taken  more  seriously  and  managed  coherently. 
[17]  When  many  thousands  of  people  must  make 
contracting  decisions,  and  wrong  decisions  can 
cost  billions,  the  need  for  quality  procurement 
training  is  ovenwhelming.  "We  need  better 
curriculum  research  technologies  to  match 
remedies  to  troubles."  [18]  The  taxonomy  could 
be  an  innovative  tool  for  matching  remedies  to 
troubles. 

Application  of  the  Taxonomy 
in  Procurement  Training 

The  first  problem  encountered  in  developing 
training  courses  is  obtaining  sufficient  task- 
descriptive  data  in  a  form  that  will  permit  the 
appropriate  design  and  conduct  of  training.  The 
procedures  the  person  is  expected  to  follow,  the 
equipment  and  tools  required,  and  the  conditions 


of  the  job  must  be  identified.  [8] 

The  taxonomy  could  be  the  tool  needed  to 
obtain  task-descriptive  data  in  an  appropriate 
form.  It  could  be  used  to  improve  procurement 
training  and  education  by  linking  and  prioritizing 
the  skills  that  are  important  in  buying  specific 
categories  of  goods.  The  procedures  the  person 
is  expected  to  follow,  the  competencies  required, 
and  the  conditions  of  the  job  can  be  more 
accurately  identified,  and  training  could  then  be 
more  accurately  tailored  to  the  student’s  needs 
The  taxonomy  would  first  identify  the  different 
categories  of  goods.  Following  classification,  the 
individual  categories  would  be  studied  to  identify 
the  specific  competencies  required  in  procuring 
those  goods.  These  competencies  encompass 
two  aspects,  both  the  task  or  skill  itself,  and  the 
degree  of  skill  required  in  performing  the  task. 
For  example,  one  buyer  may  perform  cost 
analysis  occasionally  on  a  relatively  simple  level, 
while  another  may  be  required  to  perform 
relatively  complex  cost  analysis  on  a  regular 
basis.  Both  need  cost  analysis  skills,  but  at 
different  competency  levels.  An  important  insight 
the  taxonomy  could  provide  that  is  not  otherwise 
evident  today  is  the  combination  of  skills,  and  the 
level,  appropriate  for  a  given  training  course. 

A  look  at  the  some  of  the  individual 
characteristics  may  help  demonstrate  the  insights 
that  may  be  gained  from  the  taxonomy.  The  first 
characteristic  to  consider  is  change  The 
opposite  ends  of  the  scale  for  change  are  very 
low  rate  of  technological  change  and  very  high 
rate  of  technological  change.  An  examination  of 
the  degree  of  change,  and  the  resulting 
implications,  in  the  category  of  goods  that  the 
buyer  purchases  would  allow  tailoring  of  the 
training  provided  to  that  person,  and  others  who 
procure  the  same  category  of  goods.  For 
example,  goods  with  a  very  low  rate  of 
technological  change  may  belong  to  the  simple 
category,  A  very  low  rate  of  technological 
change  could  indicate  that  competency  in 
contract  changes  is  a  low  priority.  On  the  other 
hand,  goods  with  a  very  high  rate  of  technological 
change,  possibly  the  complex  category,  may  call 
for  a  high  priority  in  contract  modification, 
configuration  change  management,  equitable 
adjustment  pricing  techniques,  and  market 
research  skills. 

Continuing  with  this  example,  the 
customization  characteristic  in  the  simple 
category  of  goods  may  be  scaled  as  no  amount 
of  customization,  and  the  complex  category  may 
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be  scaled  as  made  exclusively  for  the 
Government.  The  scale  of  no  amount  of 
customization  may  indicate  that  buyers  do  not 
need  skills  in  the  proper  selection  of 
specifications.  Indirectly,  this  may  imply  a  low 
priority  on  cost  analysis  skills,  formal  source 
selection  planning  skills,  and  negotiation  skills 
since  the  buyers  would  most  likely  be  dealing  with 
price  competition.  The  other  end  of  the 
customization  scale  would  call  for  a  high  degree 
of  competency  in  these  skills. 

Benefits  of  the  Taxonomy 
in  Procurement  Training 

It  is  not  suggested  that  the  taxonomy  of 
goods  is  the  cure-all  for  procurement  training  and 
education  problems.  It  will  not  magically 
eliminate  training  backlogs  and  increase 
resources.  However,  the  tailored  training 
approach  encouraged  by  the  taxonomy  would 
enhance  competency  and  skill  development  and 
provide  task-descriptive  data  for  designing 
courses.  Case  studies,  exercises  and  simulations 
used  in  teaching  could  be  closely  tailored  to  the 
students'  own  situations.  Innovative  techniques 
such  as  the  taxonomy,  which  assess  the 
complexity  of  goods,  are  important  in  giving  the 
issue  more  visibility. 

The  taxonomy  addresses  directly  the  need  to 
establish  training  and  education  requirements 
based  on  the  level  of  complexity  of  the  duties 
carried  out  by  individuals.  And  as  pointed  out  by 
one  of  the  contracting  experts  interviewed  for  this 
research:  "It  might  also  suggest  the  need  for 
training  in  areas  not  now  covered  by  the  more  or 
less  standard  menu  of  courses  offered." 

The  level  and  combination  of  skills. needed  in 
job  performance,  as  identified  by  the  taxonomy, 
could  be  used,  perhaps  in  conjunction  with  the 
Federal  Acquisition  institute  (FAI)  training 
blueprints,  to  develop  a  more  effective  and 
efficient  Federal  procurement  training  system. 
Viewing  the  spectrum  of  goods  in  this  manner 
would  allow  procurement  training  to  be  tailored 
directly  to  the  needs  of  the  buyers.  It  provides  a 
means  for  prioritizing  training  needs  in  order  to 
get  the  maximum  benefit  from  the  limited 
resources  that  are  available. 


RECOMMENDATIONS 

Insights  revealed  by  the  taxonomy  of  goods 
procured  by  the  Federal  Government  should  be 


used  to  develop  guidance,  not  rigid  rules.  The 
Federal  procurement  process  is  already  burdened 
with  too  many  rules  and  regulations.  The 
consequence  is  a  restrictive  procurement  process 
that  dictates  procedures  that  apply  in  all 
situations,  removing  any  room  for  flexibility  and 
individual  judgment.  The  taxonomy  should  be 
used  to  develop  guidance  and  general  principles 
that  procurement  managers  can  use  to  support 
individual  decisions. 

As  more  goods  are  classified,  application  of 
the  taxonomy  in  market  research  should  be 
refined  to  incorporate  new  insights.  The 
characteristics  of  the  goods  that  are  important  in 
this  application  may  become  evident,  requiring 
the  addition  or  deletion  of  the  characteristics  used 
for  classification.  The  taxonomy  may  help  to 
identify  information  not  yet  considered  important, 
such  as  barriers  to  competition.  The  very  nature 
of  the  way  the  Government  views  and  conducts 
market  research  may  change  as  a  result  of  further 
development  of  the  taxonomy  of  goods  procured 
by  the  Federal  Government. 

Application  of  the  taxonomy  in  procurement 
regulation  should  be  further  expanded  upon. 
Procurement  regulation  is  far-reaching  and  covers 
a  vast  portion  of  the  procurement  process.  This 
study  has  described  how  the  taxonomy  could  be 
applied  in  several  areas  of  procurement 
regulation.  There  are  many  more  aspects  of 
procurement  regulation  that  should  be  explored 
and  developed. 

Application  of  the  taxonomy  in  procurement 
training  and  education  should  be  refined  as  more 
goods  are  classified.  The  way  Federal 
procurement  training  and  education  is  conducted 
may  change  as  a  result  of  the  insights  provided 
by  application  of  the  taxonomy.  New  light  may 
be  shed  on  what  the  training  priorities  should  be. 
It  is  natural  that  the  application  of  the  taxonomy 
will  need  to  be  refined  as  new  insights  are  gained. 

Future  efforts  should  continue  to  examine 
potential  applications  of  the  taxonomy.  This 
research  effort  has  attempted  to  identify  potential 
areas  of  application  and  suggest  potential  benefits 
to  be  gained  from  the  taxonomy.  In  doing  so,  the 
general  thought  process  or  logic  of  how  the 
system  could  be  used  was  presented,  citing 
potential  implications.  This  process  should  be 
continued,  perhaps  by  taking  a  single  application, 
in  conjunction  with  a  single  category  of  goods 
and  examining  the  full  range  of  insights  that  may 
be  drawn  from  that  application. 
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FIGURE  1 


CHARACTERISTICS:  DEFINITIONS  AND  SCALES 


Complexity  describes  the  good's  technical 
intricacies.  The  degree  of  complexity  may  be  in 
terms  of  the  skill  and  expertise  needed  to 
produce  the  good.  Another  view  is  whether  the 
good  is  a  system,  sub-assembly,  component, 
piece  part,  or  raw  material.  A  1  indicates  little  or 
no  technological  complexity  with  5  being  very 
high  complexity. 

1  Very  low  technical  complexity 

2  Low  technical  complexity 

3  Medium  technical  complexity 

4  High  technical  complexity 

5  Very  high  technical  complexity 

Customization  is  the  degree  to  which  the 
good  is  manufactured  to  the  buyer’s 
specifications.  Some  goods  have  no  amount  of 
customization  while  others  are  produced 
exclusively  for  a  buyer.  Goods  that  are  not 
customized  should  be  scored  1  with  those 
developed  exclusively  for  the  Government  scored 
5. 

1  No  amount  of  customization 

2  Low  degree  of  customization 

3  Medium  amount  of  customization 

4  High  amount  of  customization 

5  Made  exclusively  for  the  Government 

Maintainability  refers  to  the  amount  of 
maintenance  considerations  associated  with  the 
good.  In  other  words,  how  frequently,  if  at  all,  is 
maintenance  is  required  on  the  good.  Some 
goods  are  virtually  maintenance-free  while  others 
require  a  great  deal  of  maintenance  throughout 
their  lives. 

1  No  maintenance  required 

2  Low  maintenance  requirements 

3  Medium  maintenance  requirements 

4  High  maintenance  requirements 

5  Very  high  maintenance  requirements 


Unit  cost  is  the  good's  cost  to  the  buyer. 
Generally  speaking,  as  a  good  becomes  more 
unique  to  the  buyer’s  requirement,  the  unit  cost  is 
increasing.  Use  1  for  low  unit  cost  and  5  for  very 
high. 

1  Very  low  unit  cost 

2  Low  unit  cost 

3  Medium  unit  cost 

4  High  unit  cost 

5  Very  high  unit  cost 

Documentation  is  another  characteristic 
external  to  the  good  yet  many  times  a  necessary 
part  of  it.  Frequently  the  Government  requires 
substantiating  documentation  in  the  form  of 
drawings,  technical  manuals,  and  certifications  for 
some  types  of  goods  while  for  others  little  at  all  is 
required.  A  1  would  indicate  a  good  purchased 
with  no  accompanying  documentation  while  5  is 
for  goods  accompanied  by  drawings,  technical 
manuals,  etc. 

1  No  associated  documentation 

2  Low  amount  of  documentation 

3  Medium  amount  of  documentation 

4  Great  deal  of  documentation 

5  Very  high  amount  of  documentation 

item  attention  given  by  the  buyer  refers  to 
single-item  versus  volume  or  mass  buying.  When 
a  buyer  deals  with  small  dollar-value  items  like 
common  bolts  and  rivets,  the  focus  is  on  a  mass 
quantity  of  these  types  of  goods.  Contrast  this 
with  the  acquisition  of  a  F-14  aircraft  where  the 
buyer's  attention  is  focused  on  a  single  item. 

1  Complete  volume-type  attention 

2  Mostly  volume-type  attention 

3  Either  volume  or  single  item  attention 

4  Usually  single-item  attention 

5  Always  single-item  attention 
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ABSTRACT 

Negotiations  play  a  significant  role  in  the 
acquisition  of  goods  and  services  not  only  in  the 
Federal  Government  but  also  within  the 
commercial  world.  The  importance  of 
procurement  negotiations  suggests  the  need  for 
a  continuing  effort  to  improve  negotiation 
effectiveness  and  thereby  improve  the  results 
attained  through  negotiations. 

This  research  examined  the  effect  on  a 
negotiated  outcome  if  buyers  engage  in 
preparatory  simulated  negotiations.  If  it  was 
found  that  simulated  negotiations  resulted  in  a 
significantly  improved  outcome  during  the  actual 
negotiation,  then  the  conduct  of  such  preparatory 
simulated  negotiations  could  enhance  negotiator 
effectiveness.  The  research  question  was;  What 
are  the  key  factors  associated  with  the  use  of 
simulations  in  preparation  for  actual  negotiations 
and  how  might  these  factors  be  used  to  enhance 
the  preparation  for  negotiation? 

One  hundred  and  thirty  nine  negotiations 
involving  (1)  Department  of  Defense  (DOD) 
buyers,  (2)  industry  contractors,  and  (3)  student 
participants  were  conducted  at  three  colleges, 
four  DOD  contracting  activities  and  four  defense 
contractors’  facilities.  The  data  collected  from 
these  negotiations  included  the  prices  negotiated 
and  a  qualitative  assessment  based  on  the 
respondents’  answers  to  a  series  of 
questionnaires. 

Based  on  an  analysis  of  these  data,  it  was 
concluded  that  buyers  engaging  in  simulated 
negotiations  improved  the  negotiation  outcome. 
The  study  also  reports  the  key  differences 
concerning  the  utility  of  simulated  negotiations 


among  the  three  test  groups. 

INTRODUCTION 

Negotiation  is  of  crucial  importance  in  DOD 
acquisitions.  The  selection,  training,  preparation 
and  performance  of  contract  negotiators  have 
been  continuing  concerns,  as  indicated  by  the 
Report  of  the  Commission  on  Government 
Procurement  in  1972.  [1]  In  a  recent  study,  the 
U.S.  Merit  Systems  Protection  Board  (MSPB) 
highlighted  the  ability  to  conduct  negotiations  as 
one  of  the  essential  areas  needing  additional 
training  in  order  to  improve  the  quality  of  the 
acquisition  workforce.  [2].  Preparation  is 
considered  by  a  number  of  authors  and 
researchers  to  be  the  key  element  of  negotiator 
effectiveness.  Simulated  negotiations  have  been 
found  to  be  potentially  prominent  among  various 
preparation  techniques.  Accordingly,  it  was  the 
purpose  of  this  research  to  explore,  through 
experiment,  the  use  of  simulated  negotiations  by 
the  buyer  and  to  determine  what  effect,  as 
measured  by  price,  the  use  of  this  technique  had 
on  the  negotiated  outcome.  Additionally,  the 
research  sought  to  identify  those  factors  in  the 
simulated  negotiation  process  which  enhanced 
the  preparation  for  actual  negotiations.  This 
research  did  not  attempt  to  determine  the  effect 
of  other  variables  of  negotiation  effectiveness, 
such  as  structural,  physical,  issue,  or  other 
negotiator  variables.  It  also  did  r.'>t  attempt  to 
measure  the  effects  of  personality  characteristics 
on  the  negotiated  outcome. 

PREPARATION 

It  has  been  said  that  at  least  90  percent  of 
success  in  negotiations  is  due  to  thorough 
preparation.  [3]  One  of  the  most  important  points 
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of  negotiation  is  that  the  team  that  obtains  more 
of  their  objectives  after  a  negotiation  is  generally 
the  one  that  was  better  prepared.  The  reverse  is 
also  true.  If  an  opponent  comes  out  best  at  the 
end  of  a  negotiation,  it  is  usually  because  the 
negotiator  was  not  prepared. 

According  to  William  F.  Morrison,  “If  a  picture 
is  worth  a  thousand  words,  then  one  role-play  will 
prevent  thousands  of  mistakes."  [3]  The  essence 
of  this  quote  is  to  practice  the  negotiation  in  order 
to  avoid  mistakes  during  the  real  thing.  Morrison 
goes  on  to  suggest  that  the  negotiator  ask  a  peer 
or  their  boss  to  take  the  part  of  the  opponent  and 
to  practice  the  negotiation  with  them  in  a  role 
play.  The  purpose  would  be  to  find  all  of  the 
weak  points  in  the  negotiator’s  plan.  He  further 
suggests  that  an  even  better  idea  would  be  for 
the  negotiator  to  find  a  person  in  their 
organization  who  performs  the  same  functions  as 
the  person  with  whom  they  will  be  negotiating. 
For  example,  if  the  negotiator  is  a  buyer,  he 
should  ask  a  seller  in  their  company  to  role-play 
with  them.  The  rationale  is  simple.  In  general, 
people  in  purchasing  do  well  in  performing  the 
role  of  a  buyer,  but  not  very  well  as  sellers 
because  they  approach  the  seller's  position  from 
a  buyer’s  perspective.  Their  value  system  is  not 
the  same.  The  converse  also  holds  true.  (3] 
Simulated  negotiations  have  long  been  used  in 
preparing  for  labor  contract  negotiations.  Dale 
Yoder  writes  that  "...  both  firms  and  unions  use 
workshops  and  practice  sessions-’simulations,’  in 
more  sophisticated  terms-with  mock  sessions 
and  role  playing  to  provide  training  and 
preparation  for  their  negotiations."  [4]  Forsini, 
Shaw  and  Blake  reported  that,  through  the 
simulated  negotiation  experience,  union 
representatives  become  aware  of  many  potential 
issues  and  controversies  that  are  not  readily 
apparent  on  the  surface  of  a  negotiating  situation. 
Thus,  they  become  forewarned  and  prepared  to 
handle  these  situations.  [5] 

Discussions  during  this  research  effort  with 
purchasing  and  marketing  representatives  from  a 
number  of  firms  who  handle  military  and 
commercial  sales  revealed  a  wide  variety  of 
techniques  when  preparing  for  negotiations. 
These  techniques  range  from  meetings  to  discuss 
and  review  a  company’s  proposal  to  the  other 
extreme  of  actually  conducting  a  "mock" 
simulated  negotiation.  The  list  below  summarizes 
some  of  the  preparatory  techniques  used  by  five 


different  manufacturing  firms  in  preparing  for 
negotiations.  While  this  list  is  by  no  means 
complete,  it  does  represent  some  of  the  more 
popular  preparatory  techniques  used  by  the 
industrial  firms  contacted. 

Preparation  Techniques 

*  Meeting  between  procurement  supervisor 
and  negotiator(s)  to  review  proposal. 

*  Publicly  available  video  tapes  on 
negotiations. 

*  Mock  negotiation  case  studies. 

*  Video  recording  of  mock  negotiation  cases. 

*  Negotiation  seminars 

*  Dry  run  of  negotiation  with  manager  playing 
the  role  of  "devil’s  advocate." 

*  Tiger  team  approach. 

*  Simulated  negotiations. 


At  a  minimum,  all  the  firms  conducted  discussions 
to  review  a  company’s  proposal.  These 
discussions  could  be  as  simple  as  deciding  the 
targets  of  the  firm  (maximum,  minimum,  and 
objective  positions)  to  mapping  out  the  strategy 
and  tactics  that  will  be  used  during  the 
negotiation.  Frequently,  these  discussions  are  the 
starting  point  in  the  preparatory  process.  Beyond 
these  discussions,  some  firms  take  the  process 
one  step  further  and  use  a  Tiger  Team  approach 
or  go  through  an  actual  dry  run  of  the  negotiation 
with  the  manager  playing  the  role  of  "devil’s 
advocate."  In  the  Tiger  Team  approach,  a  group 
of  negotiators  frequently  brainstorm  different 
strategies  and  tactics  to  be  used  and  may  also 
practice  questioning  techniques  in  an  informal 
sort  of  role  play.  Building  upon  this  approach, 
the  negotiating  team  may  conduct  a  dry  run  of 
the  negotiation.  In  the  dry  run,  the  negotiators 
begin  with  their  opening  moves  and  the  strategy 
and  tactics  the  team  intends  to  employ.  The 
manager  or  some  other  team  member  plays  the 
role  of  the  devil’s  advocate,  asking  probing 
questions  and  trying  to  find  flaws  in  the 
negotiators’  logic.  Role-plays  generate 


422 


considerable  enthusiasm  and  contribute  to 
building  a  team.  Those  in  the  role-play  generally 
exert  themselves  to  prepare  the  case  because 
they  \want  to  appear  professional  in  front  of  their 
peers.  As  a  result,  the  team  members  learn  a  lot 
about  the  process  of  negotiation.  [3]  Another 
extension  of  this  process  is  the  use  of  a  Murder 
Board  which  consists  of  sei.ior  purchasing, 
materials  management,  finance,  manufacturing, 
quality,  engineering,  and  general  management 
personnel.  Like  the  dry  run  approach,  the 
negotiating  team  presents  its  agenda,  objectives 
and  tactics  for  the  forthcoming  negotiations. 
Members  of  the  murder  board  then  dissect  the 
negotiating  plan  in  an  effort  to  identify  avoidable 
problems.  [5]  A  final  extension  of  this  preparatory 
process  is  for  the  negotiators  to  conduct  a  formal 
mock  or  simulated  negotiation  similar  to  the  one 
previously  described.  In  this  simulation,  company 
negotiators  play  the  different  buyer  and  seller 
roles  against  one  another  and  actually  go  through 
the  negotiation  from  start  to  finish.  In  some 
cases,  the  negotiation  is  video  taped  and  then 
later  reviewed  and  critiqued  by  the  group.  Video 
taping  was  found  to  be  an  extremely  valuable,  low 
cost  preparatory  tool  for  effective  negotiation. 
Use  of  the  video  system  allows  the  negotiators  to 
retrospectively  evaluate  their  strengths  and 
weaknesses,  to  evaluate  their  negotiation  strategy 
and  to  see  what  their  "body  language"  was 
saying.  In  some  cases  a  negotiator  would  be 
saying  one  thing,  thinking  they  were  coming  from 
a  position  of  power,  when  in  fact  their  body 
language  was  saying  quite  the  opposite. 

The  simulation  technique  is  valuable  in  the 
preparation  for  negotiations  because  it  allows  the 
players  to  act  out  the  entire  negotiation  before  it 
takes  place.  The  process  helps  the  negotiators 
see  what  lies  before  them  in  the  coming 
negotiation  and  presents  it  much  more  vividly 
than  if  they  merely  talked  about  it.  This  method 
also  gives  the  negotiators  a  chance  to  try 
something  without  the  risk  of  failure.  Simulated 
negotiations  permit  the  negotiator  to  bring  into 
focus  any  important  elements  that  may  have  been 
overlooked  or  ignored  in  their  original  assessment 
of  a  proposal.  Furthermore,  the  technique 
facilitates  making  corrections  in  their  preparation 
because  it  allows  the  negotiator  to  put  themselves 
across  the  table  and  see  the  other  person’s  point 
of  view  before  the  negotiation. 


Brosius  and  Erickson  conducted  an 
experiment  in  1974  to  measure  the  effect  of 
simulated  negotiations  on  final  negotiated  results. 
[61  They  used  DOD  procurement  careerists  as 
participants  and  divided  them  into  two  groups; 
experimental  and  control  They  defined  negotiated 
outcome  as  the  price  the  buyer  would  pay.  They 
statistically  compared  the  price  that  the 
experimental-group  buyers  negotiated  in  the 
“actual"  negotiation  with  the  price  that  the  control 
group  negotiated. 


RESEARCH  DESIGN 

The  design  of  the  instant  research  evolved 
from  a  field  experiment  conducted  at  the  Naval 
Postgraduate  School  (NPS)  This  field  study  was 
based  on  the  test  structure  employed  by  Dr. 
David  N.  Burt  to  measure  the  effect  of  simulated 
negotiations  as  a  preparation  technique  on 
negotiation  effectiveness.  [7]  This  study  was 
constructed  in  much  the  same  manner  as  the 
Brosius  and  Erickson  study.  Control-group 
participants  playing  the  role  of  the  buyer  (B,) 
negotiated  with  participants  playing  the  role  of  the 
seller  (S,)  in  the  "actual"  negotiation.  Control- 
group  buyers  negotiated  only  once  in  the  "actual" 
negotiation.  The  outcome  of  the  negotiation  was 
the  final  negotiated  price  of  the  contract.  Next, 
experimental-group  participants  playing  the  role  of 
the  buyer  (Bj)  engaged  in  simulated  ^gotiations 
with  participants  playing  the  role  of  the  buyer’s 
supervisor  (boss)  (Bj)  before  "actual"  negotiations 
with  the  seller  (Sj).  Analogous  to  the  control 
group,  participants  playing  the  role  of  the  seller 
negotiated  only  once  in  the  "actual"  experiment. 
One  cycle  of  the  experiment  required  five 
individuals,  A  summary  of  the  assigned  roles  is 
provided  as  follows: 

Simulated  Actual 
Group  Negotiation  Negotiation 

Control  None  B,  against  S, 

Experimental  B^  against  B3  against  Sj 


If  the  mean  price  negotiated  by  the 
participants  playing  the  role  of  Bj  was  statistically 
significant  from  the  mean  price  negotiated  by 
participants  playing  the  role  of  B,,  it  could  be 
concluded  that  simulated  negotiation  affected 
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negotiation  effectiveness,  i.e.,  price  negotiated, 
when  employed  by  the  buyer  and  not  by  the 
seller  as  a  preparation  technique  for  negotiation. 
The  instrument  used  to  generate  both  the 
simulated  negotiations  and  the  actual  negotiations 
was  a  structured,  role-playing  contract  negotiation 
case  used  by  Professor  Burt  in  his  experiment.  [8) 

Individuals  who  served  as  subjects  in  the  NPS 
experiment  were  third  quarter  contracting 
students  in  the  Contract  Pricing  and  Negotiations 
course.  Additionally,  the  experiment  used 
questionnaires  and  personal  interviews  with  the 
experimental  buyers  to  measure  their  perceptions 
of  how  simulated  negotiations  affected  their 
performance  in  the  actual  negotiation.  The 
rationale  for  the  questionnaire  was  to  obtain  a 
qualitative  measure  of  the  simulated  negotiation 
effect.  One  questionnaire  was  administered  to  the 
buyers  after  the  simulation  round.  Following  the 
actual  negotiation,  a  second  questionnaire  was 
completed,  each  buyer  was  interviewed  and  a 
buyer  group  debrief  was  conducted.  It  is  to  be 
noted  that  the  control  and  experimental  groups 
are  not  directly  linked  together  for  the  purposes  of 
comparison.  Rather,  a  number  of  independent 
control  groups  were  run  in  order  to  establish  a 
relevant  "price  range"  from  each  of  the 
negotiations.  This  served  to  eliminate  individual 
differences  between  buyers  and  sellers  in  each 
round. 

The  experiment  was  conducted  in  the 
following  sequence  of  events  with  the  roles 
defined  below: 

Hour  #1 

The  experimental  buyer  (B^)  and  the  boss(B3) 
negotiated  and  reported  the  results.  The  boss 
then  gave  feedback  to  the  buyer. 

Hour  #2 

The  control  buyer  (B,)  and  the  control  seller  (S,) 
negotiated  and  reported  the  results. 

After  the  boss  had  concluded  the  feedback 
session,  the  experimental  buyer  completed  a 
post-simulation  questionnaire  and  prepared  for 
the  "actual"  negotiation. 


Hour  #3 

Buyer  B;,  and  the  experimental  seller  (S^) 
negotiated  and  reported  the  results. 


Hour  #4 

After  Bj  and  negotiated,  the  experimental  buyer 
completed  a  post-negotiation  questionnaire  and 
participated  in  an  interview  session. 

Each  of  the  roles  were  distributed  to  the 
participants  at  least  24  hours  prior  to  beginning 
the  experiment  in  order  to  study  the  role  and 
prepare  a  negotiating  position.  Care  was  taken  to 
ensure  that  sellers  NOT  know  whether  they  were 
negotiating  with  a  control  buyer  or  an 
experimental  buyer  as  it  was  perceived  this  would 
affect  their  motivation  during  the  experiment. 


RESEARCH  EXPERIMENT 

The  selection  of  participants  to  play  the  roles 
was  accomplished  by  soliciting  participation  from 
three  different  groups:  (1)  contracting/negotiation 
students,  (2)  Government  contracting  personnel, 
and  (3)  industry  contracting  personnel  who 
interface  primarily  with  Government  organizations. 
The  student  groups  were  comprised  of 
contracting  students  at  NPS  and  students  in 
negotiation  courses  at  the  University  of  San  Diego 
and  the  University  of  Southern  California.  A  total 
of  133  students  participated  in  the  experiment. 
While  some  of  the  students  had  limited  contract 
negotiation  experience,  the  vast  majority  of  them 
had  no  formal  negotiation  experience  other  than 
in-class  exercises.  The  case  used  for  the 
experiment  was  relatively  uncomplicated  and 
could  be  easily  handled  by  even  the  novice 
negotiators.  The  military  and  commercial 
organizations  utilized  in  this  research  are  all 
located  on  the  West  Coast  in  California,  and  are 
sufficiently  large  and  experienced  in  negotiating 
DOD  contracts  to  employ  contract  negotiators, 
contracting  officers,  contract  administrators, 
and/or  cost-price  analysts  experienced  in 
negotiating  contracts.  Four  military  activities  and 
four  commercial  corporations  were  used  in  the 
study. 


ANALYSIS  OF  NEGOTIATED  PRICE 

A  total  of  226  individuals  from  the  three 
schools  (133),  four  Government  activities  (52)  and 
four  commercial  corporations  (41)  participated  in 
the  experiment.  One  hundred  thirty-nine  rounds 
of  negotiations  were  conducted.  Nineteen 
resulted  in  an  impasse  (six  in  the  simulation 
round,  nine  in  the  actual  round  and  four  in  the 
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control  round)  and  were  discarded.  The  price 
outcome  of  the  remaining  120  rounds  of 
negotiations  consisted  of  the  dollar  amounts 
negotiated  and  agreed  upon  by  each  buyer-seller 
pair,  including  the  simulated  negotiation  between 
the  buyer  and  the  boss.  All  of  the  data  are 
summarized  in  Table  I  and  include  the  mean  price 
negotiated,  the  standard  deviation,  and  the 
number  of  elements  in  each  population. 

The  results  from  the  experiment  indicate  that 
the  final  price  was  $57,483.10  less  (better  from 
the  buyer’s  perspective)  when  simulation  had 
been  used.  The  mean  price  negotiated  was 
$2,239,458.70.  and  thus  resulted  in  a  2.5% 
decrease  in  price.  Hence,  simulated  negotiations 
as  a  preparation  for  actual  negotiations  proved  to 
be  beneficial  in  this  experiment  and  improved  the 
negotiated  outcome  for  the  buyer.  However, 
when  looking  at  each  group  individually,  all  of  the 
results  were  not  the  same.  While  the  student  and 
industry  participants  (as  a  group)  who  engaged  in 
simulated  negotiations  obtained  a  lower  price 
than  their  control  group  counterparts,  the 
Government  participants  obtained  a  higher  price. 
These  results  are  summarized  in  Table  II. 

The  student  group  had  the  biggest  decrease 
in  price  (4.8%)  when  compared  to  their  control 
group  counterparts,  and  industry  participants 
achieved  a  3.0%  reduction  in  price.  For  these 
two  groups,  the  simulation  appeared  to  be  a 
beneficial  preparatory  technique.  In  contrast,  the 
Government  participants  who  engaged  in 
simulations  prior  to  their  actual  negotiation  had  a 
3.9%  increase  in  price  when  compared  to  their 
control  group.  This  suggests  that  the  benefits  of 
simulation  as  a  preparatory  technique  may  not 
be  universal.  One  possible  explanation  for  the 
dissimilar  results  between  the  groups  may  have 
been  due  to  the  difference  in  attitude  of  the 
participants.  In  general,  more  willingness  and 
even  anxiousness,  was  noted  on  the  part  of 
students  and  industrial  participants  to  see  how 
well  they  could  perform  having  done  the 
simulation.  There  also  appeared  to  be  a  degree 
of  competition  among  the  students  and  some  of 
the  industrial  participants.  These  groups 
appeared  to  have  a  keen  interest  in  seeing  how 
they  "stacked  up"  in  comparison  to  other 
participants.  There  also  appeared  to  be  an 
element  of  wanting  to  appear  fully  competent  and 
professional  in  front  of  a  peer,  an  instructor,  or  a 
supervisor.  This  is  not  to  say  that  these  elements 


were  not  present  in  any  of  the  Government 
participants.  On  the  contrary,  some  of  the 
Government  participants  appeared  to  be  very 
aggressive  and  competitive  during  the 
experiment.  However,  as  a  group,  these 
tendencies  may  have  been  mitigated  by  concern 
over  their  ever  growing  workload  or  a  feeling  that 
"the  results  were  of  little  or  no  consequence"  to 
them.  Therefore,  some  of  the  Government 
participants  may  have  given  their  second  play  in 
the  actual  negotiation  less  than  their  best  effort. 
Furthermore,  as  a  group,  the  Government 
participants  reduced  the  price  from  the  simulation 
round  to  the  actual  negotiation  by  4.1%,  the 
largest  reduction  of  any  of  the  groups.  Therefore, 
there  may  have  been  a  feeling  on  the  part  of 
these  buyers  that  the  price  was  lower  than  what 
they  received  in  the  simulation  round  and 
therefore  was  "good  enough."  One  of  the 
Government  participants  said,  "I  knew  this  price 
would  be  acceptable,  because  my  boss  and  I 
negotiated  a  higher  price  during  the  simulation. 
I  figured  that  if  I  came  home  with  anything  less 
than  that  (the  simulation  price)  would  be  good." 


ANALYSIS  OF  QUESTIONNAIRE  RESPONSES 

The  post  negotiation  questionnaire  consisted 
of  20  questions  asked  of  each  experimental  buyer 
(Bj).  A  Likert  Scale  was  used  for  the  first  14 
questions  (a  five  point  scale  ranging  from  strongly 
agree  to  strongly  disagree).  Table  III  summarizes 
the  average  response  rates  for  each  of  the  Likert 
Scale  questions  asked  regarding  the  usefulness  of 
the  simulated  negotiations..  It  is  interesting  to 
note  that  the  student  and  industry  groups  appear 
to  correlate  more  closely  than  the  Government 
group.  Intuitively,  it  was  expected  that  there 
would  be  a  higher  correlation  between  the 
industry  and  Government  groups  because  of  the 
greater  similarity  in  their  demographics. 

Table  IV  summarizes  the  most  often  cited 
responses  to  the  question  which  asked; 
Compared  to  the  simulation,  how  did  you  feel 
during  the  actual  negotiations?  The  question  had 
twelve  adjectival  qualities  and  asked  the 
respondents  if  they  felt  MORE,  EQUALLY,  or 
LESS  on  each  during  the  actual  negotiation. 

STRENGTHS  AND  WEAKNESSES 
OF  SIMULATED  NEGOTIATIONS 

Two  of  the  questions  asked  the  respondents 
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their  opinions  and  perceptions  concerning  the 
greatest  strengths  and  weaknesses  of  the 
simulated  negotiations.  These  are  summarized 
below. 

Strengths 

•  enabled  them  to  evaluate  their  overall  position 
and  revealed  weaknesses  in  their  arguments. 

•  revealed  potential  seller  arguments  and 
helped  them  anticipate  points  of  difficulty.  In 
doing  so,  it  enabled  them  to  formulate  counter 
arguments  and  to  better  anticipate  questions  in 
the  "real"  negotiation. 

•  enhanced  their  knowledge  of  the  facts  and 
how  to  deal  with  then  during  the  nego  .iion. 
One  respondent  claimed  that  the  simulated 
negotiation  helped  him,  "get  comfortable  with  the 
numbers  and  the  situation  itself." 

•  enabled  them  to  focus  in  on  issues  not 
previously  considered  and  emphasized  the 
relevant  facts.  In  some  cases,  the  simulated 
negotiation  helped  them  adjust  or  "tweak"  their 
focus  appropriately. 

•  enabled  them  to  brainstorm  different  ideas 
and  to  prepare  for  different  contingencies.  It  also 
enabled  them  to  get  alternate  strategies  for  use 
during  the  actual  negotiation. 

•  enabled  them  to  try  out  different  ideas, 
strategies  and  tactics  that  they  would  not 
otherwise  have  tried  in  a  risk  free  environment. 
The  simulated  negotiation  helped  them  to 
determine  the  effectiveness  of  their  strategy  and 
tactics.  It  also  allowed  them  to  make  mistakes 
and  to  get  feedback  from  their  mistakes. 

•  enabied  them  to  practice  what  they  were 
going  to  say  from  start  to  finish.  It  also  enabled 
them  to  express  their  ideas  to  ensure  that  they 
were  understandable.  The  simulation  was  like  a 
rehearsal  of  what  may  happen  during  the  actual 
negotiation. 

•  reinforced  the  strengths  of  their  position  and 
confirmed  their  planning.  It  also  helped  them  to 
solidify  their  arguments  and  to  clarify  their 
objectives  and  the  points  they  wanted  to  make 
during  the  actual  negotiation. 


•  gave  the  participants  additional  preparation 
which  made  them  feel  more  confident  and 
relaxed. 

•  enabled  them  to  gain  insight  based  on  their 
bosses’  experience  and  to  obtain  new  things  to 
consider.  The  ability  to  obtain  the  bosses’ 
feedback  was  also  rated  very  highly. 

Weaknesses 

•  the  boss  did  not  have  the  same  information 
as  the  seller  and,  therefore,  was  unable  to  truly 
represent  the  seller’s  position  and  to  provide 
rationale  for  the  cost  data.  The  boss  can’t 
anticipate  all  of  the  arguments  of  the  real  seller 
and  prepare  you  for  what  will  really  happen. 

•  it  lacked  realism  and  was  somewhat  contrived 
and  artificial.  Players  could  make-up  information 
as  they  went  along  to  suit  their  purposes  without 
consequence.  The  boss  did  not  have  a  stake  in 
the  outcome. 

•  the  scenario  had  too  little  information  to  make 
it  truly  meaningful.  There  was  a  general  lack  of 
information  and  relevance  of  information. 

•  the  simulation  is  not  a  perfect  tool  because 
the  boss  has  basically  the  same  information  as 
the  buyer  going  into  the  negotiations.  Therefore, 
the  boss  already  knows  tha  buyer’s  position  and 
bottom  line.  This  knowledge  gives  the  boss  an 
unfair  advantage.  There  is  no  mechanism  to 
establish  a  legitimate  alternate  viewpoint. 

•  it  was  a  time  consuming  process  that 
interfered  with  actual  work. 

•  the  simulation  will  not  be  effective  unless  the 
person  playing  the  seller  is  motivated  to  go 
through  the  process. 

•  common  assumptions  made  by  the  buyer 
and  the  boss  might  affect  the  expectations  of  the 
buyer  in  the  actual  negotiation  and  potentially 
"cloud"  the  buyer’s  judgment.  In  other  words, 
erroneous  assumptions  made  during  the 
simulation  could  negatively  predispose  the 
negotiator  prior  to  the  actual  negotiation. 

•  simulation  was  difficult  to  take  seriously 
because  the  buyer  knew  that  the  boss  was  not 
the  seller  and  therefore  did  not  have  the  same 
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information. 

•  all  the  simulation  and  preparation  in  the  world 
will  not  help  if  the  seller  is  not  willing  to  negotiate. 

•  knowing  my  boss  was  judging  my 
performance  during  the  simulation  and  that  my 
next  raise  was  on  the  line  made  me  uneasy  in 
attempting  to  perform  well. 

A  final  question  during  the  post-negotiation 
questionnaire  asked:  Regardless  of  the  price  you 
negotiated  in  the  actual  negotiation,  do  you  feel 
that  you  did  a  better  job  of  negotiating  because 
you  had  done  the  simulation?  This  question  was 
designed  to  measure  the  respondents'  belief  that 
they  were  able  to  negotiate  more  effectively 
during  the  actual  negotiation,  specifically  asking 
if  it  was  the  simulation  which  led  to  this  improved 
performance.  Table  V  presents  the  results  of  this 
question.  The  overwhelming  response  to  this 
question  was  an  affirmation  that  the  simulation  did 
improve  their  preparedness  in  the  actual 
negotiation. 


CONCLUSIONS/RECOMMENDATIONS 

In  general,  simulated  negotiations  employed 
by  the  buyer  improves  negotiation  effectiveness 
as  measured  by  price.  The  results  of  this 
research  indicate  that  simulated  negotiations  as  a 
preparation  for  actual  negotiations  proved  to  be 
beneficial  to  the  outcome.  Although  the  research 
focuses  on  the  buyer’s  perspective,  simulated 
negotiations  should  reap  similar  benefits  for 
negotiators  representing  sellers. 

Some  of  the  participants  noted  that  the 
simulated  negotiation  process  did  have 
weaknesses.  These  results  suggest  that  the 
success  or  the  effectiveness  of  the  simulated 
negotiation  process  is  tied  directly  to  the 
motivation  and  attitudes  of  the  participants.  If  the 
participants  view  the  process  as  a  positive  and 
enriching  experience  that  will  better  prepare  them 
for  contract  negotiations,  then  chances  are  good 
that  the  simulated  negotiation  process  will 
improve  their  performance.  If,  on  the  other  hand, 
the  participants  view  the  process  as  just  another 
requirement  to  be  fulfilled  before  the  actual 
negotiation,  then  there  is  strong  evidence  to 
suggest  that  the  process  will  be  of  little  value  and 
may  in  fact  be  dysfunctional. 


A  majority  of  the  participants  felt  that  the 
simulated  negotiation  helped  improve  their  overall 
performance  in  the  actual  negotiation.  Over  95% 
of  the  participants  agreed  that  the  simulation  had 
indeed  helped  them  to  negotiate  more  effectively 
in  the  actual  negotiation.  This  ovenwhelming 
majority  lends  credibility  to  the  simulated 
negotiation  process  as  an  effective  preparatory 
technique  for  contract  negotiations.  Furthermore, 
the  results  from  the  questionnaires  indicate  that  a 
majority  of  the  participants  agreed  that  simulated 
negotiations  were  of  value  and  helped  them: 

•  evaluate  their  negotiation  strategy  and  tactics 

•  clarify  and  emphasize  the  relevant  facts 

•  focus  on  the  real  issues 

•  articulate  and  solidify  their  arguments 

•  identify  new  issues  not  previously  considered 

•  improve  their  line  of  inquiry 

•  anticipate  questions 

•  identify  the  seller’s  position 

•  try  new  strategies,  tactics  and  ideas  with  little 
risk 

In  addition,  a  strong  majority  of  the 
participants  indicated  that  they  would  like  to 
conduct  more  simulations  in  preparation  for  future 
contract  negotiations.  Beyond  these  positive 
statements  concerning  the  value  of  simulated 
negotiations,  a  majority  of  the  participants 
indicated  that  they  felt  MORE  on  the  following 
adjectival  qualities:  confident,  relaxed,  focused, 
prepared,  motivated,  creative,  and  knowledgeable. 
In  addition,  a  majority  of  the  participants  also 
indicated  that  they  felt  LESS  frustrated.  A 
majority  of  the  participants  felt  the  simulation  was 
a  worthwhile  exercise  and  was  a  very  valuable 
preparatory  technique. 

The  simulated  negotiation  process  should  be 
incorporated  into  the  professional  training  of 
Government  and  industry  contracting  personnr' 
and  should  be  integrated  more  into  graduate  and 
undergraduate  level  courses  that  deal  with  the 
contract  negotiatioti  process.  For  the  students, 
especially,  the  simulated  negotiation  served  as  an 
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additional  training  ground  and  helped  make  them 
feel  more  prepared  and  confident  concerning  their 
positions.  This  improved  confidence  can  translate 
into  improved  performance  in  the  actual 
negotiation.  Therefore,  this  kind  of  preparation  in 
the  curriculum  of  these  contracting  students  could 
have  a  significant  impact  on  the  future 
development  and  professionalism  of  tomorrow’s 
contracting  personnel  by  providing  them  greater 
insight  into  the  negotiation  process. 

The  point  is  clear  that  the  simulated 
negotiation  process  has  the  potential  to  save 
organizations  millions  of  dollars  on  their  contracts. 
Furthermore,  the  simulated  negotiation  process  is 
not  just  limited  to  the  price  of  the  contract.  The 
improvements  achieved  through  the  process 
could  just  as  well  extend  to  the  other  terms  and 
conditions.  Finally,  the  simulated  negotiation  has 
the  additional  benefit  of  conveying  to  upper 
management  the  buying  team’s  objectives  and 
tactics. 

The  simulated  negotiation  process  should  be 
constructed  in  order  to  minimize  the  amount  of 
required  resources  and  at  the  same  time 
maximize  the  effectiveness  of  the  process.  It  is 
important  to  recognize  and  to  drive  the  point 
home  that  the  simulated  negotiation  process  is 
not  without  associated  costs  and  drawbacks. 
Engaging  in  a  simulated  negotiation  is  a  time 
consuming  process.  Therefore,  there  exists  a 
tradeoff  between  the  amount  of  time  and  the 
costs  associated  with  performing  a  simulation  and 
the  amount  of  benefits  that  will  be  derived  from 
the  process  in  the  form  of  improved  negotiated 
outcomes.  And  again,  it  must  be  emphasized 
that  the  simulated  negotiation  process  is  no 
guarantee  that  an  individual  or  a  team  of 
negotiators  will  achieve  their  desired  outcomes. 
The  process  appears  to  be  a  function  of  an 
individual’s  motivation  and  attitude  towards  the 
process  and  therefore,  it  must  be  used 
judiciously  and  with  discretion.  Given  the  costs 
and  drawbacks  of  doing  a  simulated  negotiation, 
it  is  appropriate  that  it  be  used  in  those  situations 
where  it  appears  warranted  (contracts  with  a  high 
dollar  value,  major  or  important  contracts  that  will 
have  long  term  ramifications  for  the  organization, 
and  complicated  contracts  with  difficult  issues 
that  involve  tradeoffs)  and  with  people  who  are 
sufficiently  motivated  and  inclined  to  make  the 
process  work. 
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TABLE  I 


SUMMARY  DATA  OF  PRICES  NEGOTIATED 


CONTROL 

SIMULATION 

ACTUAL 

PRICE 

PRICE 

GROUP 

Mean 

Mean 

Mean 

(Std  Dev'n) 

(Std  Dev’n) 

(Std  Dev’n) 

n=  pop  size 

n=  pop  size 

n=  pop  size 

2,296,9451.80 

2,313,319.30 

2,239,458.70 

TOTAL 

(119,091.57) 

(165,872.41) 

(150,675.50) 

n  =  31 

n=46 

n=43 

2,306,791  10 

2,278,500.00 

2,195,800.00 

STUDENT 

(132,750.68) 

(147,810.10) 

(99,463.73) 

n=23 

n=23 

n=21 

2,226,750.00 

2,413,168.30 

2,313,731.00 

GOVERNMENT 

(45,350.72) 

(177,7070.87) 

(201,233.50) 

n  =  4 

n=13 

n=12 

2,310,500.00 

2,263,600.00 

2,242,015.00 

INDUSTRY 

(29,338.54) 

(130,527.54) 

(132,759.65) 

n  =  4 

n=10 

n  =  10 

TABLE  II 


COMPARISON  OF  ACTUAL  PRICES  NEGOTIATED 


GROUP 

COMPARISON  TO  SIMULATION 

COMPARISON  TO  CONTROL 

TOTAL 

$73,860.60  less 

3.2%  decrease 

$57,483.10  less 

2.5%  decrease 

STUDENT 

$82,700.00  less 

3.6%  decrease 

$110,991.10  less 

4.8%  decrease 

GOVERNMENT 

$99,437.30  less 

4.1%  decrease 

$86,981 .00  more 

3.9%  increase 

INDUSTRY 

$21,585.00  less 

1 .0%  decrease 

$68,485.00  less 

3.0%  decrease 
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TABLE  III 


SUMMARY  OF  LIKERT  SCALE  QUESTIONS 


SIMULATED  NEGOTIATIONS: 

TOT 

STU 

GOV 

IND 

1 .  Helped  evaluate  negotiation  strategy. 

4.30 

4.44 

3.79 

4.63 

2.  Helped  evaluate  negotiation  tactics. 

4.29 

4.44 

4.07 

4.18 

3.  Helped  focus  on  the  real  issues. 

4.25 

4.26 

4.07 

4.45 

4.  Helped  solidify  arguments. 

4.39 

4.40 

4.29 

4.45 

5.  Helped  identify  ne\«  issues. 

4.12 

4.19 

4.00 

4.09 

6.  Helped  improve  line  of  inquiry. 

4.23 

4.33 

3.86 

4.45 

7.  Caused  change  in  strategy  and  tactics. 

3.54 

3.48 

3.71 

3.45 

8.  Caused  change  in  Min,  Max  and  Objective 
targets. 

3.44 

3.59 

2.86 

3.82 

9.  Were  valuable  as  a  preparatory  technique. 

4.46 

4.63 

4.21 

4.36 

10.  Increased  comfort  level  with  strategy  and 
tactics. 

4.48 

4.44 

4.43 

4.64 

1 1 .  Helped  anticipate  questions. 

4.10 

4.26 

3.79 

4.09 

12.  Helped  Identify  seller’s  position. 

3.71 

3.70 

3.43 

4.J9 

13.  Helped  improve  overall  performance. 

4.35 

4.44 

4.29 

4.18 

14.  Have  encouraged  me  to  use  again. 

4.39 

4.52 

3.93 

4.64 

TOT  =  Total 
STU  =  Students 
GOV  =  Government 
IND  =  Industry 
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TABLE  iV 


SUMMARY  OF  ADJECTIVAL  QUALITIES 


TOTAL 

STUDENT 

GOVERNMENT 

INDUSTRY 

Confident 

MORE 

MORE 

MORE 

MORE 

Anxious 

EQUALLY 

EQUALLY 

LESS 

LESS 

Bored 

EQUALLY 

LESS 

LESS 

EQUALLY 

Relaxed 

MORE 

MORE 

MORE 

MORE 

Time  Pressured 

EQUALLY 

LESS 

EQUALLY 

LESS 

Focused 

MORE 

MORE 

MORE 

MORE 

Prepared 

MORE 

MORE 

MORE 

MORE 

Resentful 

LESS 

EQUALLY 

LESS 

LESS 

Motivated 

MORE 

MORE 

MORE 

MORE 

Creative 

MORE 

MORE 

MORE 

MORE 

Knowledgeable 

MORE 

MORE 

MORE 

MORE 

Frustrated 

LESS 

LESS 

LESS 

LESS 

TABLE  V 


SIMULATED  NEGOTIATIONS  LED  TO  BETTER  ACTUAL  NEGOTIATIONS 


TOTAL 

STUDENT 

GOVERNMENT 

INDUSTRY 

YES 

96% 

96% 

93% 

100% 
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Acquisition  for  Experimental  or  Test  Purposes  (AETP) 


Patricia  Lemmer  and  LtCol  Les  Katahara,  United  States  Air  Force 


ABSTRACT 


Ever  had  an  idea  for  a  product  you  encountered  in  the  commercial  markets 
but  been  frustrated  at  the  acquisition  red  tape  that  seems  to  preclude  (or 
greatly  delay)  actual  use  by  our  forces,  even  for  a  limited  test?  Ever 
needed  some  test  data  on  commercially  available  technologies  to  help 
support  a  program  decision,  but  were  unable  to  delay  for  a  "full-up" 
competition?  These  are  but  two  scenarios  that  might  radically  change  if 
the  acquisition  team  utilizes  EXISTING  statutory  authority  at  10  U.S.C. 
2274  or  10  U.S.C.  9504  which  covers  Acquisition  for  Experimental  or 
Test  Purposes  (AETP).  These  statutes  authorize  acquisition  (or  lease)  of 
aircraft,  aircraft  parts,  aeronautical  accessories,  ordnance,  signal,  and 
chemical  warfare  supplies  and  designs  thereof,  with  or  without 
competition. 

Currently,  the  AETP  statutes  are  not  widely  used  (having  been  largely 
limited  to  Foreign  Weapons  Evaluation  Program  (FWEP)  buys).  AETP 
authority  could  streamline  the  acquisition  of  necessary  hardware  and 
supplies  for  the  test  and  evaluation  portion  of  the  Integrated  Weapon 
System  Management  (IWSM)  spectrum.  The  Aeronautical  Systems  Center 
(ASC)  has  issued  a  local  AETP  policy  based  on  the  statutes  at  10  U.S.C. 
2274  and  9504. 

The  AETP  statutes  offer  promise  of  more  efficient  utilization  of 
dwindling  government  and  contractor  manpower,  can  save  taxpayers 
money,  allow  quicker  technology  insertion,  and  greater  flexibility  to 
initiate  test  programs  for  commercially  available  products/technology 
because  competition  is  not  required  for  the  initial  experiment  or  test. 

The  use  of  AETP  statutes  at  ASC  in  FY  93/94  will  be  closely  monitored. 
Quantitative  measures  of  the  actual  savings  in  lead-time  and  resources 
should  demonstrate  the  value  of  judicious  utilization  of  AETP  throughout 
DOD. 
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INTRODUCTION  TO  AETP 

Background.  Have  you  ever  had  an  idea  of 
how  the  Department  of  Defense  (DOD)  could 
use  a  commercial  product,  but  given  up  in 
frustration  due  to  mountains  of  "red  tape"  that 
seem  to  preclude  or  delay  actual  use  of  the 
commercial  product  for  even  a  limited  test 
period?  Have  you  ever  needed  some  test  data 
from  commercially  available  technologies  to 
help  support  a  program  decision  but  were 
unable  to  wait  for  a  "full-up"  competition? 
Have  you  ever  seen  an  opportunity  to  make 
significant  improvements  to  internal  office 
practices  by  utilizing  rapidly  advancing  office 
automation  commercial  technologies,  but 
wanted  to  test  the  interface  with  other  current 
computer/communication  systems  to  see  if  the 
payoff  was  really  there?  These  scenarios 
might  radically  change  if  the  acquisition  team 
utilizes  existing  statutory  authority  at  10 
U.S.C.  2274  or  9504.  (See  Figure  1) 

The  statutes  known  as  Acquisition  for 
Experimental  or  Test  Purposes  (AETP), 
provide  an  invaluable  strategy  to  allow 
simplified  acquisition  for  test  or  experiment. 
We  have  all  experienced  Defense 
Management  Review  changes,  acquisition 
reform,  manpower  and  budget  reductions  -  it 
seems  these  and  other  initiatives  have 
complicated  our  acquisition  business.  This  is 
no  news  to  most  of  us.  AETP  can  simplify 
the  acquisition  of  items  needed  for 
experiment  or  test.  Metrics  and  measurement 
of  our  processes  using  lead-time  measures 
seem  to  point  an  accusative  finger,  as  if  to 
say,  "It  takes  too  long  to  do  the  simple  and 
what  makes  common  sense."  At  ASC,  our 
1X0,000  FY  92  contractual  actions  range  from 
funding  actions  averaging  lead-times  of  1 1 
days,  to  major  source  .selections  in  501  days, 
to  definitive  contracts  under  $5M  in  57  day.s. 
But,  contracting  lead-time  is  only  one  part  of 


the  story.  Consider  the  true  lead-time  to 
deploy  a  specific  technology  to  the  field  and 
we  see  it  can  take  years  or  decades!  AETP 
can  shorten  both  lead-times  and  use  less 
resources  doing  it. 

The  Aeronautical  Sy.stems  Center  (ASC)  at 
Wright-Patterson  AFB,  OH  has  issued  a  local 
AETP  policy  based  on  the  statutes  at  10 
U.S.C.  2274  and  9504.  The  AETP  statutes 
authorize  acquisition  (or  lea.se)  of  aircraft; 
aircraft  parts;  aeronautical  accessories; 
ordnance,  signal,  and  chemical  warfare 
supplies;  (and  designs  thereof)  for 
experimental  or  te.st  purposes,  with  or  without 
competition!  Interested?  Read  on! 

Any  new  acquisition  strategy  must  1)  improve 
the  Air  Force  mission  and  2)  satisfy  the 
overall  intent  of  the  Competition  in 
Contracting  Act  (CICA)  statute  -  that  is  to 
promote  competition  in  the  most 
MEANINGFUL  way.  So,  the  AETP  story 
can  be.st  be  understood  by  looking  at  the 
legislative  history  and  historical  application 
of  the  AETP  statutes. 

Legislative  History,  Section  2274  of  Title 
10,  United  States  Code,  Procurement  for 
experimental  purposes,  provides  in  pertinent 
part: 

(a)  The  Secretary  of  a  military 
department  may  buy  designs,  aircraft, 
aircraft  parts  and  aeronautical 
acces.sories  that  he  considers  necessary 
for  experimental  purposes  in  the 
development  of  the  best  kinds  of  those 
items  for  the  Army,  Navy,  Air  Force, 
or  Marine  Corps,  as  the  case  may  be. 
Purchases  may  be  made  abroad  or  in 
the  United  States  or  the  Territorie.s, 
commonwealths,  and  possessions, 
with  or  without  competition  and  by 
contract  or  otherwise. 
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Section  9504,  Procurement  for  experimental 
purposes  provides  in  pertinent  part: 

The  Secretary  of  the  Air  Force  may 
buy  ordnance,  signal,  and  chemical 
warfare  supplies,  including  parts  and 
accessories,  and  designs  thereof,  that 
he  considers  necessary  for 
experimental  or  test  purposes  in  the 
development  of  the  best  supplies  that 
are  needed  for  the  national  defense. 
Purchases  under  this  section  may  be 
made  inside  or  outside  the  United 
States,  with  or  without  competitive 
bidding,  and  by  contract  or  otherwise. 

The  former  statute  originated  in  legislation 
originally  passed  as  part  of  the  Air  Corps  Act 
of  July  2,  1926.  The  bill  apparently  was 
originally  conceived  over  concern  that  United 
States  air  power  lacked  superiority  in  the  post 
World  War  1  era.  The  proposed  bill  was  an 
attempt  to  address  the  air  superiority  issue  by 
providing  more  flexibility  to  procure 
aeronautical  items  for  experimental  purposes 
at  home  or  abroad.  Production,  however,  was 
to  be  done  by  domestic  enterprises  only.  This 
legislation  has  since  been  codified  at  10 
United  States  Code  §  2274. 

In  1930  the  War  Department  became 
concerned  over  the  time  and  cost  required  to 
develop  equipment  and  supplies  for  the 
national  defense.  As  a  result  a  bill  expressly 
designed  to  extend  the  provisions  of  the 
original  legislation  was  enacted  to  cover 
purchases  of  signal,  ordnance,  and  chemical 
warfare  items  for  experimental  or  test 
purposes.  This  legislation  is  now  codified  at 
10  United  States  Code  §  9504. 

The  Competition  in  Contracting  Act  (CICA) 
was  passed  in  19X4,  requiring  full  and  open 
competition  unless  an  acquisition  falls  under 
certain  enumerated  exceptions.  ^  Thus,  on  its 
face,  CICA  may  appear  to  directly  contradict 


the  above  statutes.  Nonetheless,  the  rules  of 
statutory  construction  disfavor  implicit  repeal 
of  earlier  laws.  Clear  evidence  of  Congress' 
intent  to  implicitly  repeal  earlier  statutes  is 
required  to  support  such  a  position.  Such 
evidence  is  absent  in  the  in^tant  case.  In  fact, 
CICA  explicitly  provides  for  exemptions 
from  full  and  open  competition  when 
"expressly  authorized  by  statute."  It  is, 
therefore,  clear  that  these  statutes  survive  the 
Competition  in  Contracting  Act  and  are 
currently  available  to  .support  such 
procurement  without  competition. 

AETP 

Thesis.  This  paper  will  communicate  1 )  the 
existence  of  AETP  statutes  to  the  DOD 
acquisition  community  and  2)  the  sensible 
implementation  policy  developed  at  ASC.  It 
will  defend  the  following  thesis:  AETP 
statutes  at  10  U.S.C.  2274  and  9504  offer 
more  efflcient  utilization  of  dwindling 
government  and  contractor  resources,  can 
save  taxpayers  money,  allow  quicker 
technology  insertion,  and  greater  flexibility 
to  conduct  test  programs  for  commercially 
available  products  or  technology,  and  are 
likely  to  generate  more  meaningful 
competition  in  the  future  as  technologies 
and  products  are  proven  to  be  useful  fur 
DOD. 

Lastly,  a  warning  to  those  who  hope  AETP 
will  solve  all  of  your  problems.  AETP  is  not 
a  panacea  -  it  is  not  appropriate  to  acquire 
production  quantities  or  for  acquisitions 
outside  the  experiment  or  test  phase. 

Genesis  of  AETP  Policy.  The  opening 
chapter  of  the  AETP  story  at  ASC  was 
written  by  a  process  review  team  which  met 
in  June  1990  to  improve  and  streamline  the 
PRAM/RAMTIP  contracting  process.  These 
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two  ASC  offices  are  chartered  to  insert 
existing  and  emerging  technologies  into  Air 
Force  aircraft,  missiles,  and  support 
equipment.  The  Productivity,  Reliability, 
Availability,  and  Maintainability  (PRAM) 
Program  Office  inserts  mature,  off-the-shelf 
technology  into  existing  weapon  system.s, 
support  equipment,  and  maintenance  depot 
operations.  The  Reliability  and 

Maintainability  Technology  Insertion 
Program  (RAMTIP)  Office  focuses  on 
emerging  technologies  that  can  be  applied  to 
current  systems  and  those  under  development. 
In  1990,  PRAM  and  RAMTIP  recognized  a 
need  to  reduce  the  time  it  takes  to  insert 
technologies:  PRAM  programs  took  36 

months  to  go  from  an  idea  to  a  fielded 
system,  and  more  complex  RAMTIP 
programs  took  48  months!^  This  need  to 
reduce  PRAM/RAMTIP  lead-times  indirectly 
resulted  in  the  AETP  policy.  The  outcome  of 
the  PRAM  process  review  team  effort  was  six 
specific  recommendations,  one  of  which  was 
to  develop  a  new  CICA  exception  to  Full  and 
Open  Competition  in  the  DFARS  Part 
206.302-1  to  allow  the  test  and  evaluation  of 
domestic  systems  and  equipment  (similar  to 
the  exception  that  exi.sts  for  the  Foreign 
Comparative  Testing  Program).^ 

Chapter  two  of  AETP  opens  with  the 
preparations  in  Nov  90  to  develop  a  FAR  case 
for  the  new  CICA  exception  for  testing  of 
domestic  items.  The  first  step  was  to  check 
the  data  that  the  process  review  team  had 
generated.  Among  the  documentation  was  a 
note  with  the  two  statutory  references  and 
some  old  Secretary  of  the  Air  Force  (SAF) 
and  Air  Force  Systems  Command  (AFSC) 
staffing  documents  discussing  these  same 
statutes.  Next,  a  trip  to  the  law  library  to  read 
the  cited  statutes.  They  were  amazingly  short 
and  concise.  At  that  point,  the  realization 
that  ASC  did  not  need  to  develop  a  FAR  ca.se 
to  allow  us  to  process  J&As  under  CICA  -  we 
already  had  statutory  authority  to  acquire 


items  required  for  experiment  or  test  (both 
foreign  and  domestic)!  The  "contracting 
types"  readily  agreed  that  the  two  statutes 
appeared  to  provide  the  basis  for  a  "new" 
acquisition  authority  and  that  we  ought  to 
research  it  and  then  issue  a  local  policy. 
Next,  it  was  necessary  to  tell  the  AETP  .story 
to  the  attorneys  and  see  if  they  found  our 
conclusion  defensible. 

Chapter  three  started  with  a  reque.st  for  a  legal 
opinion  of  the  AETP  statutes,  their  legislative 
history,  and  any  potential  conflict  with  the 
CICA  .statute.  In  Sept  1991,  the  ASC/JAG 
legal  opinion  was  received.  It  stated  in  part, 
"There  is  in  our  opinion,  no  direct  conflict 
between  10  U.S.C.  2274,  10  U.S.C.  9504,  and 
CICA.'"^  The  legal  opinion  also  stated  that 
delegation  of  authority  from  the  Secretary  had 
taken  place.^  It  was  decided  that  the  package 
outlining  the  local  policy  would  be  sent  to 
legal  and  contracting  at  HQ  AFSC  for  their 
review.  The  ASC  policy  was  refined  and 
coordinated  internally  with  contracting,  legal, 
and  PRAM  personnel.  In  early  1992,  it  was 
decided  to  issue  the  local  policy  and  then 
notify  the  new  HQ  Air  Force  Materiel 
Command  (AFMC)  after  the  merger  of  the 
old  Logistics  and  Systems  Commands  was 
complete.^  Internal  ASC  coordination  was 
completed  on  1 3  Nov  92  and  PK  policy  letter 
92-054  was  issued  to  the  ASC  contracting 
workforce.^ 

ASC  Policy  and  Procedures.  The  ASC 
policy  letter  provides  guidance  on  the 
background  of  the  AETP  statutes  and 
examples  of  proper  u.sage  or  application.  It 
emphasizes  that  work  for  research  and/or 
development  of  equipment,  designs,  etc.  or 
work  other  than  experimental  or  te.st  purposes 
is  not  allowable  using  AETP.  Several 
scenarios  are  briefly  described  which  we 
believe  are  permissible  under  AETP 
authority:  test  quantities  of  items:  some 
te.sting  services  (limited):  and  minor 
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modifications  to  equipment  for  test  purposes, 
etc.  Until  substantial  experience  is  gained  at 
ASC,  use  of  AETP  authority  will  require  a 
Determination  and  Findings  (D&F)  executed 
by  the  Program  Manager,  Contracting 
Officer,  Staff  Judge  Advocate,  and  the 
Competition  Advocate  (only  if  over  $5M). 
(See  Figure  2)  Our  local  procedures  guarantee 
a  legal  review  and  coordination  on  every 
application  of  the  AFTP  authority; 
Competition  Advocate  staff  review  on 
significant  (over  $2M)  actions  with 
Competition  Advocate  approval  when  over 
$5M;  and  an  annual  report  to  ASC/PKC  of  all 
acquisitions  where  AFTP  authority  has  been 
used,  in  addition  to  the  policy  letter,  the  ASC 
FAR  Supplement  part  17  was  modified  to 
clearly  detail  AFTP  authority,  application.s, 
and  procedures.^ 

It  is  hoped  that  valid  measurements  can  be 
developed  to  assess  savings  in  lead-time  and 
resources  to  demonstrate  the  value  of 
judicious  utilization  of  AFTP  throughout 
DOD.  Realistically,  it  will  take  the  remainder 
of  FY  93  and  FY  94  to  gain  significant 
experience  utilizing  AFTP  at  ASC. 

Potential  ASC  Applications.  AFTP 
applications  are  likely  to  be  small  one  time 
buys  or  lease,  and  are  restricted  to  limited 
quantities  of  items  or  supplies  needed  for 
experimental  or  test  purpo.ses.  Di.spo.sable 
items  may  require  thousands  of  items  for 
testing  over  a  defined  time  period,  whereas, 
larger,  complex  items  may  be  purchased  or 
lea.sed  in  quantities  of  only  one. 

Examples  of  situations  where  AETP  authority 
is  appropriate  include;  limited  quantities  of 
equipment  or  supplies  needed  for 
governmental  testing  of  commercial  items  in 
the  field  (as  in  the  PRAM/RAMTIP  mi.ssion); 
Prime  Mission  Equipment  (PME)  or  existing 
military  hardware  required  by  test  programs 
directed  by  PMDs  or  other  established 


requirements;  items,  parts,  or  equipment 
required  for  item  qualification  testing  as 
required  in  FAR  part  9.2;  demonstrations  of 
compatibility  of  computer-communications 
equipment  with  pre-existent  AF  computer- 
communications  systems;  non-developmental 
item  advocacy;  minor  modifications  to 
commercially  available  items  for  test 
purposes,  safety  issues,  fit,  or  test  data 
collection;  or  design  studies  that  result  in  a 
specific  design/methodology  to  be  used  in  test 
and  experimentation.  Testing  .services  may 
be  covered  only  in  limited  situations  as  ASC 
has  interpreted  that  AETP  authority  is  not 
intended  to  cover  general  testing  services. 

Based  on  review  of  recent  J&A  files  where  a 
CICA  exception  to  Full  and  Open 
Competition  was  obtained,  it  appears  that 
ASC  may  process  approximately  30  AETP 
acquisitions  annually.  This  number  might  be 
larger  if  small  source  selections  are 
eliminated  in  favor  of  AETP  authority.  It 
al.so  depends  on  the  impact  of  several 
organizations  now  included  in  ASC  as  a  re.sult 
of  the  addition  of  the  old  Wright-Patterson 
Contracting  Center.  It  is  difficult  to  estimate 
with  much  certainty  the  number  of  ASC  uses 
of  AETP. 

Non-ASC  Applications.  ASC  is  the  only  AF 
organization  known  to  be  using  AETP  and  a 
quick  check  with  Army  &  Navy  Competition 
Advocates  didn’t  indicate  that  they  utilize  the 
AETP  statutes. 

Technology  Transfer.  Although  the  AETP 
statutes  are  not  to  be  used  for  true  R&D,  it  is 
conceivable  that  they  will  be  u.seful  for 
technology  transfer  from  both  the  commercial 
sector  AND  government  laboratories  into 
DOD  systems.  Limited  duration  testing  of 
these  technologies  can  be  accomplished 
quickly  and  provide  feasibility  data  to  our 
Program  Managers.  As  defense  laboratories 
work  to  reorient  their  research  focus  to 
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commercial  applications  of  technological 
breakthroughs,  smarter  and  quicker 
acquisition  strategies  will  be  important. 
That  is  the  real  payoff  of  AETP.  After  the 
AETP  testing,  any  production  quantities  will 
be  required  to  comply  with  the  CICA 
requirement  for  competition. 

Lease.  The  lease  aspects  of  AETP  are  based 
on  the  fact  that  the  original  statutes  use  the 
terms  "procurement"  and  "buy"  which  aren't 
defined  in  the  FAR.  We  have  equated 
"procurement"  with  the  current  term  of 
acquisition  to  include  both  purchase  and 
lease.*^  Also,  we  look  at  lease  as  a  temporary 
purchase  that  makes  the  most  sen.se  for  high 
dollar  equipment  for  a  limited  experiment  or 
test  period. 

Issues  Remaining.  The  final  chapters  of  the 
AETP  story  have  yet  to  be  written.  Other 
organizations  will  now  have  a  hand  in 
determining  how  the  AETP  story  ends.  ALso, 
.several  challenges  remain  to  be  worked  at 
ASC.  To  fully  utilize  these  statutes  we  need 
to  "Get  the  word  out."  The  policy  letter  has 
been  issued  in  the  contracting  community  but 
there  remains  .some  training  to  increase 
awareness  within  the  acquisition  community. 
Development  of  specific  metrics  to  assess  the 
lead-time  reductions  and  savings  in  resources 
has  yet  to  be  done.  Common  sense  tells  us 
that  these  are  real  benefits  but  we  would  like 
to  quantify  them.  Color  of  money  (i.e.  R&D 
Vs.  production  money)  should  not  be  an  issue 
since  we  believe  it  is  appropriate  to  u.se  R&D 
funds  for  AETP  testing. 

CONCLUSION 

The  AETP  .statutes  at  10  U.S.C.  2274  and 
9504  hold  great  promise  of  savings  in 
resources,  shorter  lead-times,  and  quicker 
technology  in.sertion  because  experimental 
or  test  efforts  may  be  contracted  for  quickly 


and  directly.  Competition  is  enhanced  in  the 
long-term  because  managers  will  be  able  to 
obtain  test  data  needed  for  decision  making 
on  requirements  for  subsequent  development 
or  production.  The  use  of  AETP  statutes  at 
ASC  by  SPOs.  Operational  &  Specialized 
Contracting,  Wright  and  Armament 
Laboratories,  and  for  PRAM/RAMTIP 
projects  in  FY  93/94  will  be  monitored. 
Quantitative  measures  of  the  actual  savings  in 
lead-time  and  resources  should  demonstrate 
the  value  of  judicious  utilization  of  AETP 
throughout  DOD. 

Additional  information  concerning  the  AETP 
.statutes  and  the  ASC  policy  implementing 
them  may  be  obtained  from  Patricia  Lemmer. 
ASC/PKCA  Bldg.  14,  1X65  Fourth  St  Ste  X, 
Wright-Patterson  AFB  OH  45433-7122, 
telephone  (513)255-4192  or  DSN  7X5-4192. 
Lt  Col  Le.ster  Katahara  can  be  reached  at 
AFCLC/JANA  Bldg  11,  2240  B  St  Ste  X, 
Wright-Patterson  AFB  OH  45433-7122. 
telephone  (5 1 3 )255-527{),x22 1 . 

The  authors  would  like  to  thank  the 
contracting  officers  who  helped  to  shape  this 
AETP  policy,  the  process  review  team  and 
PRAM  who  first  identified  the  need,  and  the 
others  who  offered  suggestions  and 
encouraged  us!  This  paper  reflects  the  views 
of  the  authors  and  does  not  reflect  the 
opinions  of  ASC  as  a  whole. 
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2274.  Procurement  for  experimental  purposes 

(a)  The  Secretary  of  a  military  department  may  buy  designs,  aircraft,  air¬ 
craft  parts,  and  aeronautical  accessories  that  he  considers  necessary  for  ex¬ 
perimental  purposes  in  the  development  of  the  best  kinds  of  those  items  for 
the  Army,  Navy,  Air  Force,  or  Marine  Corps,  as  .the  case  may  be. 

Purchases  under  this  subsection  may  be  made  abroad  or  in  the  United  States 
or  the  Territories,  Commonwealths,  and  possessions,  with  or  without  com¬ 
petition  and  by  contract  or  otherwise. 

(b)  If,  as  a  result  of  a  purchase  under  subsection  (a),  a  new  or  suitable 
design  considered  to  be  the  best  kind  for  the  Army,  Navy,  Air  Force,  or 
Marine  Corps,  as  the  case  may  be,  is  developed,  the  Secretary  of  the  military 
department  concerned  may  contract  for  procurement  of  the  item  in  quanti¬ 
ty.  Contracts  under  this  subsection  are  subject  to  sections  2272(f)  and  2279 
of  this  title  but  are  not  subject  to  sections  2271(a)  (d)  and  2272(a)  of  this 
title. 

(Aug.  10,  1956,  C.  1041, 70A  Stat.  126.) 


9504.  Procurement  for  experimental  purposes 

The  Secretary  of  the  Air  Force  may  buy  ordnance,  signal,  and 
chemical  warfare  supplies,  including  parts  and  accessories,  and 
designs  thereof,  that  he  considers  necessary  for  experimental  or  test 
purposes  in  the  development  of  the  best  supplies  that  are  needed  for 
the  national  defense.  Purchases  under  this  section  may  be  made 
inside  or  outside  the  United  States,  with  or  without  competitive 
bidding,  and  by  contract  or  otherwise.  Chapter  137  of  this  title 
applies  when  such  purchases  are  made  in  quantity.  Aug.  10,  1956, 
c.  1041.  70A  Stat.  575. 


AETP  Statutes  from  10  U.S.C. 


Figure  1 


439 


DEPARTMENT  OF  THE  AIR  FORCE 
DETERMINATION  AND  FINDINGS 


Authorization  to  Enter  into  a  Contract  Under  the 
Acquisition  for  Experimentation  and  Test  Purposes  Statutes 

Upon  the  basis  of  the  following  findings  and  determination,  the  proposed 

contract/lease  for  _  (item  name  and  quantity) _ may  be 

entered  into  pursuant  to  the  authority  of  10  U.S.C.  2274  and  /or  9504. 

FINDINGS 


1.  Statement  of  Need: 

_ (Description  of  technical  attributes  of  item(s) 

_ that  will  be  tested  for  military  applications.) 


2.  Technical  Determination  of  Optimal  Technology/Item: 

_ (Selection  of  name  brands,  P/N,  or  process  to  be  tested.). 


3.  Method  of  Test/Experimentation  Description  and  Quantity  to  be  Tested: 


DETERMINATION 

The  proposed  contract/lease  for  _ (dollar  value) _  may  be  entered  into 

pursuant  to  the  AETP  statute(s). 

I  certify  the  above  to  be  true. 

Requirements  Office  signature  Legal  Coordination 

Contracting  Officer  signature  Competition  Advocate  (over  S5M) 

ASC  FAR  SUPPLEMENT  Figure  5317.96-2  -  AETP  Determination  and  Findings 

Figure  2 
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The  Concept  of  Value  in  Government  Contracting 


by  Robert  E.  LJcyd 
U.S.  Department  of  State* 


ABSTRACT 

The  Defense  Management  Review 
(DMR)  has  been  a  major  force  for 
change  in  Defense  acquisition  in 
recent  years.  One  of  its  fundamental 
principles  is  that  contracting  rules 
should  be  limited  to  only  those 
procedures  and  practices  which  "add 
value."  Regrettably,  there  has  been 
no  formal  definition  of  the  term 
"value"  to  guide  the  intended  changes 
in  the  acquisition  system,  even 
though  this  term  has  long  been  the 
source  of  a  lively  debate  as  to  its 
meaning.  This  paper  reviews  the 
varied  definitions  of  value  in  an 
effort  to  reach  an  understanding  of 
the  concept  that  will  provide  a 
standard  by  which  to  judge  the 
results  of  the  DMR. 

The  concept  of  value  appears  in 
a  variety  of  different  contexts  in 
Government  contracting,  including 
"best  value"  acquisition  and  value 
engineering.  Value  has  also  been 
studied  extensively  in  other  fields 
such  as  philosophy  and  economics, 
where  writers  have  regarded  the 
principle  of  value  in  ways  as  varied 
as  subjective  preferences  (as  in 
axiology),  a  measure  of  utility  (as 
in  Austrian  economics),  or  even  as  a 
function  solely  of  labor  applied  (as 
in  the  Marxist  labor  theory  of 
value) . 

After  an  analysis  of  this  array 
of  definitions,  an  attempt  is  made  to 
provide  a  baseline  for  judging  future 
acquisition  policy  and  its 
implementation.  The  paper  concludes 
that  value  is  a  weak  criterion  by 
which  to  judge  acquisition  policy, 
and  unless  a  consistent  view  of  value 
is  adopted,  the  DMR  implementation  in 
the  future  may  allow  purely 
subjective  uses  of  this  concept  to 
perpetuate  the  same  deficient 
management  practices  that  originally 
prompted  the  DMR. 


*The  views  and  opinions  expressed  are 
solely  the  author's  and  do  not 
necessarily  represent  those  of  the 
U.S.  Government. 


INTRODUCTION 

The  "Joint  OSD-DOD  Component 
Regulatory  Relief  Task  Force"  formed 
pursuant  to  the  DMR  conducted  a 
complete  review  of  all  Department  of 
Defense  (DOD)  acquisition-related 
guidance  with  an  eye  toward 
streamlining  such  policies  and 
procedures  and  focusing  on  practical 
use.  The  task  force  report  concluded 
with  its  own  guidance  that  "the  need 
for  a  policy,  procedure,  or 
requirement  must  clearly  provide 
value  added  to  our  fundamental 
mission."'  DOD's  fundamental 

mission  is  to  provide  the  military 
forces  needed  to  deter  war  and 
protect  the  security  of  our  country.^ 
The  problem  is  that,  while  this 
mission  may  be  clear,  the  Task  Force 
did  not  define  the  meaning  of 
"value".  This  is  not  surprising.  A 
uniform,  satisfactory  definition  of 
the  term  has  never  been  developed. 
For  excunple.  Hall's  250  page  book 
titled  What  is  Value?  concluded  that 
an  answer  cannot  be  formulated.^ 
Smart  notes  that  "the  history  of 
economic  science  is  strewn  with 
wrecks  of  theories  of  value."*  By 
reviewing  these  value  theories  we 
will  be  able  to  discern  whether  the 
use  of  "value  added"  as  a  criterion 
for  acquisition  policy  provides  a 
sound  basis  to  guide  policy  makers. 

LAW  AND  REGULATION 

The  Federal  Acquisition 
Regulation  (FAR)  contains  three 
explicit  references  to  "value": 

•  FAR  15.605(c)  regarding 

best  value  contracts; 

•  FAR  15.812(c)  concerning 

the  value  added  by 
noncompetitive  supply 
contractors;  and 

•  FAR  Part  48  on  value 

engineering  (VE). 

In  each  case,  the  FAR  envisions  a 
different  concept  of  value,  although 
in  no  case  is  the  term  explicitly 

defined. 
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The  first  reference  states 
that,  in  certain  acquisitions,  the 
Government  may  select  the  source 
whose  proposal  offers  the  greatest 
value  to  the  Government  in  terms  of 
performance  and  other  factors. 
Several  authors  have  attempted  to 
define  this  as  "best  value,"  meaning 
acquisitions  where  price  factors  are 
not  the  primary  determinant  of  who 
receives  the  contract  award.  Best 
value  decisions  include  technical, 
performance,  and  price  consider¬ 
ations.^  Greatest  value  has  also 
been  defined  simply  as  a  type  of 
award  decision  in  which  technical 
factors  are  considered  more  important 
than  price.*  While  such  definitions 
are  relatively  straightforward  and 
unambiguous,  they  provide  little  help 
for  the  acquisition  manager  in 
determining  whether  to  issue  a  policy 
or  procedure.  Certainly  both  cost 
and  technical  factors  must  be 
considered,  but  beyond  that  there  is 
no  clear  standard  in  this  portion  of 
the  FAR  for  deciding  when  an 
acquisition  policy  "adds  value." 

The  second  citation  is  based  on 
the  only  statutory  reference  to  value 
in  procurement  law  (10  U.S.C. 
2304 ( i) (2 ) ( B) ) ,  which  requires  that, 
in  the  case  of  a  noncompetitive 
supply  contract,  the  contractor  must 
identify  those  supplies  it  did  not 
manufacture  or  to  which  it  did  not 
contribute  "significant  value."  This 
rule  is  presumably  for  the  purpose  of 
enabling  the  Government  to  negotiate 
better  prices  by  seeking  to  reduce 
the  indirect  costs  or  profit 
associated  with  subcontracted  items. 
Again,  value  is  not  defined,  but  it 
appears  to  mean  labor  or  similar 
production  effort  when  dealing  with 
so-called  value-added  resellers 
(VARs) .  As  will  be  discussed  below, 
somehow  it  seems  incongruous  for  DOD 
to  be  relying  on  the  equivalent  of  a 
labor  theory  of  value  in  its  contract 
relations  with  capitalist  industry. 

The  third  area  listed  above, 
VE,  features  relatively  lengthy 
coverage  in  the  FAR  compared  to  the 
other  two  citations,  but  still  no 
definition  of  value  is  provided. 
Nevertheless,  the  literature  on  value 
engineering  provides  ample  discussion 
of  the  matter,  and  a  consensus 
appears  to  have  been  reached  on  a 
definition  which  resembles  that  used 
for  greatest  value  acquisition;  VE 


writers  consistently  describe  value 
as  a  relationship  between  function  or 
performance  and  cost. 

The  founder  of  VE,  Larry  Miles, 
defines  value  as  appropriate 
performance  and  cost.  Value  can  be 
increased  by  decreasing  costs  or 
increasing  performance.’  Jerry 
Kaufman  provides  the  clearest 
definition  of  value:  function  divided 
by  cost.  He  notes  that  value  as 
viewed  by  the  producer  is  the  ratio 
of  function  to  cost,  but  as  viewed  by 
the  buyer  means  perceived  benefits 
divided  by  price.  This  seemingly 
clear  definition  masks  a  larger 
complexity.  To  gauge  value  by 
comparing  benefits  to  price  is 
essentially  to  engage  in  traditional 
cost-benefit  analysis,  which  has  long 
been  criticized  for  its  weaknesses  in 
a  normative  sense  as  well  as  its 
inadequacy  in  treating  intangible 
factors.*  To  assess  value  in  terms  of 
function  is  likewise  difficult  to  do 
objectively;  Kaufman  defines  function 
as  the  intent  or  purpose  of  a  product 
or  system,  operating  in  its  normally 
prescribed  manner,  i.e.,  anything 
that  makes  an  item  or  system  work  or 
sell.  He  insists,  however,  that 
function  cannot  be  quantitatively 
expressed.’ 

At  least  one  other  VE  author 
has  elaborated  on  this  last  point. 
Fallon's  VE  text,  quoting  Frondizi, 
asks  whether  values  have  to  be 
necessarily  objective  or  subjective; 
he  views  absolute  definitions  as 
missing  the  point,  for  value  is 
inherently  both  objective  and 
subjective.'®  Our  search  for  a 
definition  of  value  which  can  readily 
serve  as  a  benchmark  for  acquisition 
policy  makers  must  therefore  go 
beyond  the  boundaries  of  VE  and  the 
FAR. 

AXIOLOGY  AND  SUBJECTIVE  DEFINITIONS 

One  of  the  chief  problems  with 
the  term  "value"  is  that 
unfortunately,  as  Rescher  notes,  "in 
the  English  language  the  word  is  used 
in  a  somewhat  loose  and  fluctuating 
way.""  Value  has  been  variously 
described  as  personal  bias, 
attribution,  empirical  quality, 
logically  indefinable,  self- 
realization,  a  qualified  object  of 
interest,  and  so  on.”  Mudge's  study 
of  value”  notes  that  "Value  is  a 
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difficult  word  to  define  because  it 
is  applied  to  both  subjective 
qualities  (those  determined  by  the 
emotions)  and  objective  qualities 
(those  determined  by  external 
characteristics)."  Of  all  the 
standards  to  choose  for  judging 
acquisition  policy,  value  is  surely 
one  of  the  least  impressive.  The 
established  field  of  philosophical 
study  )cnown  as  axiology,  which  deals 
with  the  general  theory  of  value,  has 
sought  a  unified  philosophy  of  value 
but  is  generally  regarded  as 
unsuccessful .  '■* 

Axiology  has  been  dominated  by 
a  dichotomy  between  objective  and 
subjective  value.  For  example, 
Aristotle  named  seven  classes  of 
value;  economic,  moral,  aesthetic, 
social,  political,  religious,  and 
judicial,  of  which  only  the  economic 
was  considered  objective  and 
measurable  in  external  units.  Yet 
even  within  economic  value  can  be 
found  both  subjective  and  objective 
qualities:  use  value  (properties  that 
accomplish  a  use,  work,  or  service; 
esteem  value  (properties  that  make 
ownership  of  an  object  desirable); 
cost  value  (properties  which  are  the 
sum  of  the  labor,  material,  overhead, 
and  other  costs  to  produce 
something);  and  exchange  value 
(properties  of  an  object  that  make  it 
possible  to  procure  other  things  by 
trading).  A  single  outlook  on  value 
can  be  misleading.'' 

Rescher  claims  that  a  value 
is  no  more  than  a  slogan  capable  of 
providing  rationalization  for  action 
by  reflecting  a  positive  attitude 
toward  a  supposedly  beneficial  state 
of  affairs.'*  This  makes  value 
little  more  than  personal  preference 
masquerading  as  objective  criterion. 
An  adequate  theory  of  value  should 
account  for  both  aspects  of  value: 
some  things  are  valued  because  they 
are  regarded  by  people  in  a  certain 
way,  and  other  things  because  they 
are  valuable,  i.e.,  objectively 
possessed  of  certain  value-endowing 
features.  To  exclude  subjectivity 
from  value  is  impossible.  Value  must 
be  assessed  not  abstractly  but  in 
context . 

Edel's  survey  of  value  theory'^ 
also  asserts  that  there  is  no  unity 
of  definition  when  discussing  value. 
In  this  century,  the  generic  concept 


of  value  was  so  vague  that  it  could 
encompass  preferential  tendencies  of 
people  in  any  field,  because  the  term 
had  just  enough  ambiguity  to  connote 
both  a  state  of  liking  and  criteria 
for  evaluation.  American  social 
scientists  focused  on  value  studies, 
but  the  concept  of  value  was  not 
clarified;  value  was  often  used  as  a 
judgmental  reference,  as  the  criteria 
employed  for  evaluation  of 
conduct/decision  or  for  justifying 
norms.  By  Pareto's  time,  efforts  to 
compare  values  across  individuals  had 
been  abandoned,  and  valuation  was 
considered  the  expression  of 
preferences  aimed  at  overall 
maximization.  Value  thus  character¬ 
ized  any  interest  that  motivates  a 
person.  Although  Dewey  refined  the 
definition  of  value  as  selection  and 
rejection  rather  than  simply  desire, 
in  an  effort  to  form  a  more  neutral 
term  (less  reminiscent  of  hedonism), 
the  subjectivity  inherent  in  the  word 
remains  inescapable. 

Perhaps  the  most  prominent 
recent  philosophical  work  on  value 
theory,  David  Gauthier's  Morals  by 
Agreement,  comes  down  squarely  on  the 
side  of  treating  value  as  a 
subjective,  relativistic  measure  of 
individual  preference.  "Value  is  not 
an  inherent  characteristic  of  things 
or  states  of  affairs ...  independent  of 
persons  and  their  activities." 
Gauthier  argues  that  the  best 
explanation  for  human  action  is  that 
"choice  maximizes  preference 
fulfillment  given  belief."  Values 
are  not  objective  in  this  theory,  but 
instead  value  is  "subjective  because 
it  is  a  measure  of  preference  and 
relative  because  it  is  a  measure  of 
individual  preference.""* 

The  results  of  axiology  have, 
like  the  regulatory  uses  of  value, 
failed  to  produce  a  consistent, 
meaningful  standard  of  value  on  which 
to  judge  acquisition  policy.  To  base 
policy  on  personal  preferences  (as 
much  value  theory  amounts  to)  is  to 
continue  regulating  contracts  as  we 
have  in  the  past,  without  any  change 
in  substance.  Before  concluding  that 
subjective  analysis  is  unavoidable, 
however,  we  should  consider  the 
chances  for  discovering  useful 
criteria  in  another  field:  economics. 
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ECONOMICS  AND  OBJECTIVE  DEFINITIONS 

Perhaps  the  greatest  effort  at 
defining  value  has  been  made  by 
economists.  A  leading  dictionary  of 
economics'’  defines  value  as  the 
intrinsic  worth  of  a  commodity.  If 
defined  in  terms  of  money,  value 
determines  price.  Traditionally 
there  have  been  separate  concepts  of 
use  value  and  value  in  exchange; 
value  in  use  is  the  capacity  to 
satisfy  human  wants,  while  value  in 
exchange  is  the  worth  of  a  commodity 
in  terms  of  its  capacity  to  be 
exchanged  for  another  commodity. 
Classical  economists  required  the 
existence  of  use  value  or  utility/ 
usefulness  for  an  item  to  be 
exchanged;  exchange  value  was 
considered  to  be  determined  by  the 
cost  of  production  (wages,  profit, 
and  rent).  Although  the  so-called 
"labor  theory  of  value"  is  best  known 
from  the  works  of  Karl  Marx,  it 
actually  has  its  origin  in  Adam 
Smith's  Wealth  of  Nations.  Smith 
argued  that  labor  is  the  true  measure 
of  the  value  of  commodities,  because 
what  something  is  worth  to  someone  is 
the  "toil  and  trouble  which  it  can 
save  to  himself,  and  which  it  can 
impose  upon  other  people."^ 

Ricardo  further  advanced  the 
labor  theory  of  value,  arguing  that 
the  value  of  a  commodity  depends  on 
the  relative  quantity  of  labor 
necessary  for  its  production,  though 
this  held  true  only  under  certain 
special  conditions.  In  the  Ricardian 
system,  the  theory  of  value  was  a 
real-cost  theory,  with  labor  as  most 
important  empirical  factor.  The 
labor  theory  of  value,  as  modified, 
was  deemed  the  best  general 
explanation  of  relative  prices;  the 
relation  between  value  and  labor-time 
is  that  every  increase  in  the 
quantity  of  labor  must  augment  the 
value  of  the  commodity  on  which  it  is 
exercised,  but  the  value  of  some 
commodities  is  due  to  scarcity  alone. 

In  the  late  nineteenth  century, 
neoclassical  economists  focused  on 
market  price  rather  than  the 
intrinsic  value  analyzed  in  the  labor 
theory  of  value.  They  viewed  price 
as  determined  by  marginal  utility  of 
a  commodity  and  hence  the  source  of 
exchange  value.  Marshall's  supply 
and  demand  curves  were  derived  from 
utility;  the  theory  of  value  became  a 


theory  of  allocation  of  scarce 
resources  to  specific  uses  rather 
than  a  search  for  intrinsic  value. 
Modern  theory  continues  this  line, 
with  its  dual  incentives  of  utility 
maximization  for  consumer  (demand) 
and  profit  maximization  for  producer 
(supply),  based  on  the  unifying 
principle  of  marginal  utility. 

Only  Marxists  retained  the 
separation  of  value  theory  (labor 
theory  of  value  or  theory  of 
profit/surplus  value)  from  the  theory 
of  price  determination.  Sowell  notes 
that  to  Marx,  value  was  the  units  of 
society's  labor,  the  underlying 
essence  and  the  relations  among 
commodities,  while  exchange  value  is 
only  a  surface  appearance.’’  To 
Marx,  labor  was  the  essence  of  all 
value,  and  value  was  an  objective 
property  of  every  commodity.  He  was 
not  impressed  by  subjective 
interpretations  of  value  based  on 
utility,  since  he  was  a  materialist; 
the  objective  common  element  in  all 
commodities  was  labor.  Market  prices 
were  not  random  but  fluctuated  around 
a  definite  point,  the  cost  of 
production  based  on  labor,  so  value 
was  determined  not  by  laws  of  the 
market  but  by  production  itself." 

This  line  of  thinking,  while 
intriguing,  has  no  application  to 
contemporary  acquisition  policy,  for 
to  argue  that  any  application  of 
labor  in  the  development  of  a 
contracting  procedure  adds  value  is 
to  provide  no  standard  at  all  and 
could  lead  to  a  nearly  infinite 
increase  in  regulations. 
Furthermore,  Nozick's  trenchant 
critique  of  the  labor  theory  of  value 
points  out  that  Marx's  central  notion 
of  socially  necessary  labor  time  is 
itself  defined  in  terms  of  the 
processes  and  exchange  ratios  of  a 
competitive  market.  This  analysis 
reveals  the  tautology  inherent  in 
Marxist  value  theory.^ 

Perhaps  the  most  ambitious 
explorations  in  value  theory  were 
made  by  the  so-called  Austrian  school 
of  economics,  principally  Eugen  von 
Bohm-Bawerk,  who  once  again 
distinguished  two  types  of  value: 
subjective  and  objective.  In  his 
value  theory,  some  objects  have 
intrinsic  value,  while  others  are 
considered  valuable  only  as  a  means 
to  an  end:  economic  value.  In  its 
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subjective  sense,  value  denotes  the 
significance  which  a  good  possesses 
for  the  well-being  of  a  certain 
subject,  meaning  that  possession  of 
this  good  satisfies  some  want, 
provides  some  gratification,  affords 
some  pleasure,  or  spares  some  pain, 
which  one  would  be  forced  to  forego 
if  one  did  not  possess  the  good.  The 
presence  of  a  good  means  a  gain  for 
one's  well-being.  The  other  kind  of 
value  is  objective,  signifying  our 
estimate  of  the  capacity  of  a  good  to 
bring  about  some  definite  extrinsic 
objective  result.  This  defines  value 
as  an  appraisal  of  the  relationship 
that  exists  between  the  good  and  the 
accomplishment  of  some  objective 
purpose  or  result.  The  phenomenon  of 
value  thus  has  a  dual  nature. 

Bbhm-Bawerk' s  formal  definition 
of  value  is  the  significance  which  a 
good  possesses  for  promoting  the 
well-being  of  an  individual.  All 
goods  have  usefulness,  but  not  all 
goods  have  value;  in  order  for  value 
to  exist,  usefulness  must  be  paired 
with  scarcity.  Any  value  that  exists 
at  all  is  concrete;  there  is  no 
abstract  categorical  value.  The 
value  of  a  good  is  determined  by  its 
marginal  utility,  and  the  magnitude 
of  marginal  utility  depends  on  the 
relation  between  wants  and  the 
wherewithal  to  satisfy  them. 
Usefulness  and  scarcity  are  the 
ultimate  determinants  of  value. In 
the  over-regulated  world  of 
Government  contracts,  a  cynic  might 
argue  that  the  best  way  to  increase 
the  value  added  by  today's 
acquisition  policies  is  to  make  them 
as  scarce  as  possible. 

Another  celebrated  Austrian 
economist,  Carl  Menger,  was  even  more 
insistent  on  the  subjective  character 
of  value.  He  argued  that  if 
economizing  individuals  perceive  that 
the  satisfaction  of  one  of  their 
needs  is  dependent  upon  their  command 
of  a  good,  then  this  good  attains  for 
them  the  significance  we  call  value. 
The  value  of  goods  springs  from  the 
same  source  as  the  economic  character 
of  goods,  i.e.,  the  relationship 
between  requirements  for  and 
available  quantities  of  goods.  Value 
is  not  inherent  in  goods  but  merely 
the  importance  that  we  attribute  to 
the  satisfaction  of  our  needs.  That 
a  good  has  value  to  us  is 
attributable  to  the  fact  that  command 


of  it  has  for  us  the  significance  of 
satisfying  a  need  that  would  not  be 
provided  for  if  we  did  not  have 
command  of  the  good. 

The  value  of  goods  arises  from 
their  relationship  to  our  needs  and 
is  not  inherent  in  the  goods 
themselves.  It  is  erroneous,  Menger 
argued,  to  speak  of  values  as 
independent  real  things  and  to 
objectify  value  in  this  way. 
Objectification  of  the  value  of  goods 
without  relation  to  their  importance 
to  economizing  individuals  is 
subjective  in  nature  and  causes 
confusion.  Only  satisfactions  have 
importance  for  us;  value  is  imputed. 
The  measure  of  value  is  entirely 
subjective  in  nature,  and  for  this 
reason  a  good  can  have  great  value  to 
one  individual,  little  value  to 
another,  and  no  value  to  a  third, 
depending  on  the  differences  in  their 
requirements  and  available  amounts. “ 

If  marginal  utility  determines 
value,  though,  then  value  is  solely  a 
matter  of  whose  perception,  personal 
preferences,  or  utility  are  at  stake, 
which  is  hardly  a  meaningful  standard 
for  the  development  of  acquisition 
policy.  Returning  to  the  dictionary 
again,  "value-added"  is  defined  as 
the  value  of  the  firm's  output  minus 
the  value  of  the  inputs  it  purchases 
from  other  firms. ^  In  a  Government 
contract  context,  this  means 
subtracting  the  prime  contractor's 
subcontract  costs  from  the  contract 
price  to  the  Government  to  determine 
if  any  "value"  was  added.  The 
regulatory  analogy  of  adopting  such  a 
principle  would  be  to  require  policy 
makers  to  estimate  the  worth  of  the 
output  and  ensure  that  this  exceeded 
the  cost  of  the  inputs  (drafting  and 
implementing  a  regulation) ,  somewhat 
akin  to  a  cost-benefit  analysis 
(although  the  DMR  Task  Force  wisely 
did  not  impose  any  such  requirement). 
Consequently,  this  seemingly 
objective  definition  once  again 
conceals  a  more  complex  situation. 

As  mentioned  above,  the 
standard  methodology  for  cost-benefit 
analysis  has  been  subjected  to 
serious  question  in  terms  of  its 
legitimacy  as  a  basis  for  policy.  If 
acquisition  policy  is  intended  to 
maximize  the  benefit/cost  ratio  for 
DOD's  mission,  this  must  involve 
measurement  of  preferences  of  both 
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the  individuals  performing  contracts 
and  those  who  rely  on  contracts  for 
implementation  of  their  programs. 
Such  measurement  is  both  practically 
and  technically  difficult  and  can 
also  be  skewed  by  individuals  who 
misrepresent  their  preferences.  For 
example,  different  policy  drafters 
may  have  different  goals,  reflecting 
either  personal  biases  or  an 
underlying  statutory  ambiguity  which 
is  subject  to  varying  interpretation 
by  different  analysts.  Some  may  seek 
efficiency  while  others  advocate 
internal  controls  or  fairness  and 
equity  in  the  treatment  of 
contractors  as  a  guiding  principle. 

In  addition,  the  measurement 
process  generally  falls  short  in 
addressing  intangible  costs  and 
benefits,  as  well  as  failing  to 
account  for  the  concerns  of  equity. 
The  practical  problem  is  best  summed 
up  by  Paris  and  Reynolds: 

"How  close  can  the  analyst  come 
to  making  accurate  assessment 
of  the  costs  and  benefits  of 
alternative  policy  proposals? 
If  cost-benefit  analysis  is  to 
be  intellectually  sound,  it  is 
necessary  to  assess  the  full 
range  of  costs  and  benefits 
associated  with  each 
alternative.  The  sheer  number 
of  effects  is  likely  to  make 
this  a  difficult  task.  It  is 
compounded  by  the  fact  that 
many  effects  may  be  unintended 
and  thus  hard  to  identify,  much 
less  evaluate."” 

Measurement  decisions  in  cost-benefit 
analysis  may  clearly  reflect 
subjective  or  ideological 
determinations.  Further,  to  include 
externalities  and  intangibles  in  the 
analysis  means  that  their  assessment 
"may  tend  to  have  the  status  of 
highly  subjective  guesses. 

This  sentiment  is  echoed  in  a 
recent  work  by  Formaini,  who  uses  the 
learning  of  Austrian  economics  to 
challenge  the  very  notion  of 
scientific  cost-benefit  analysis  by 
pointing  out  that,  due  to  the  lack  of 
objective  data  regarding  both  costs 
anr)  hf.nefivs,  no  claim  can  be  made 
for  the  rationality  of  a  public 
policy  or  undertaking.  He  concludes 
that  all  policy  is  normative  in 
nature.”  Our  hope  for  objective 


criteria  for  value  theory  in  the  form 
of  cost-benefit  analysis  has  thus 
proved  fruitless. 

CONCLUSION 

Even  if  we  are  unable  to  remove 
all  subjective  elements  from  value 
theory,  there  may  yet  be  standards 
which  are  reasonably  objective, 
recognizing  that  we  will  never  attain 
complete  objectivity.  In  the  search 
for  a  less  ambiguous  standard,  we 
might  look  to  the  statutory  goal  of 
efficiency*  in  trying  to  develop 
examples  of  how  to  make  policy  which 
promotes  efficiency.  Such  examples 
might  include  reducing  lead-time  for 
contracting,  eliminating  paperwork 
burdens,  or  obtaining  better  prices 
or  performance.  It  cannot  be 
expected,  however,  that  standards 
such  as  these  will  be  available  as 
criteria  in  most  cases  where  a  new 
acquisition  policy  is  being 
contemplated.  In  addition,  focusing 
solely  on  efficiency  will  cause  the 
exclusion  of  policies  enacted  for 
reasons  such  as  internal  control  or 
fairness  to  contractors.  The 
relatively  objective  standard  of 
efficiency  is  itself  problematic, 
which  means  we  should  face  up  to  the 
relativism  and  duality  inherent  in 
any  attempt  to  use  value  as  a 
criterion  for  judging  policy. 
Perhaps  it  is  best  to  abandon  the 
pretense  of  objectivity  and 
impartiality  and  recognize  that 
acquisition  policy  is  as  susceptible 
to  abuse  as  any  other  aspect  of 
Government  contracting  reform. 

A  review  of  the  various 
definitions  and  meanings  of  value 
points  to  a  definite  conclusion:  in 
seeking  a  basis  for  evaluating  the 
wisdom  of  acquisition  policy,  one 
would  be  hard  pressed  to  choose  a 
less  helpful  standard  than  value. 
The  concept  of  value  has  a  long 
history  of  difficulty  in  its 
definition;  in  fact,  if  any  consensus 
has  been  reached  at  all  in  the  field 
of  value  inquiry,  it  is  that  it  is 
impossible  to  separate  the  objective 
and  subjective  elements  of  value. 
Despite  the  DMR's  intention  to 
instill  a  greater  sense  of  discipline 
and  rationality  in  Defense 
acquisition  management,  the 
consequence  of  judging  policy  by  the 
fuzzy  standard  of  "value  added" 
cannot  be  to  increase  the 
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effectiveness  of  acquisition  policy¬ 
making.  While  efficiency  may  offer 
some  occasional  usefulness  as  a  tool 
in  deciding  which  acquisition 
policies  and  procedures  to  issue,  the 
concept  of  using  "value  added"  as  a 
basis  for  deciding  the 
appropriateness  of  a  policy  is  a 
misguided  idea. 
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Abstract 

This  paper  summarizes  key  concepts  de¬ 
veloped  over  seventeen  years  toward  a  two- 
stage  capstone  concept  tor  increasing  pro¬ 
ductivity  substantially  in  an  Integrated-DSM 
(Defense  Systems  Management)  or  IDSM 
process  that  covers  the  fife  cycle  of  every 
defense  ^stem.  The  central  first-stage 
question  is:  How  to  provide  every  Defense 
Systems  Acquisition  Management  (DSAM) 
professional  better  access  to  the  huge,  widefy 
distributed,  bocfy  of  DSAM  knowledge, 
whenever  the  professional  needs  relevant 
DSAM  knowledge  and  information  (K&I)? 

The  two-stage  capstone  is  a  concept  for 
long-term  evolutionary  development  of  a 
DSAM  then  an  IDSM  K&I  Locator  System. 
The  system  would  increasingly  enable  pro¬ 
fessionals,  first  in  DSAM  then  in  IDSM 
centers  of  management  excellence  through¬ 
out  their  respective  commututies  to  access 
relevant  DSAM/IDSM  K&I  when  needed  in 
a  job  or  for  a  task  at  hand. 

The  concept,  when  implemented,  will 
increase  productivity  of  professioruils,  first 
throughout  the  DSAM  commuruty,  then 
throi^out  the  IDSM  community,  and  tiius, 
increase  ^oductivity  substantially  in  a 
seamlessly  integrated  IDSM  process  covering 
every  defense  system’s  entire  life  cycle. 

Introduction 

This  paper  concerns  productivity,  and 
systems  concepts  for  increasing  productivity. 
Implementing  these  concepts  substan- 
tiaUy  increase  DSM  process  productivity  of 
two  groups  of  professionals,^  DSAM  profes¬ 
sionals,  and  future  IDSM  professionals. 

The  DoD  should  act  now  to  begin  using 
new  Information  Management  Technologies 
(IMT)2  to  implement  these  concepts.  First, 
the  action  wfil  enable  DSAM  professionals 
throughout  the  DSAM  community,  to  in¬ 
crease  productivity  substantially  in  the 
DSAM  process.  Ultimately,  it  will  enable  a 
larger  group  of  IDSM  professionals  through¬ 
out  the  IDSM  conununity,  to  increase  pro¬ 


ductivity  in  tire  IDSM  process  even  more 
tiiroughout  every  system’s  entire  life-cycle. 

Background.  How  to  manage  acquisition 
of  defense  systems  most  effectively  has  al¬ 
ways  been  a  serious  problem.  As  defense 
systems  became  more  complex,  the  acquisi¬ 
tion  management  problem  became  increas- 
ingty  difficult  and  more  visible,  prompting 
new  DoD  policy-makers  to  improve 
management  of  systems  acquisition. 

In  tire  early  1960s,  Defense  Secretary 
McNamara  instituted  the  Total  Package  Pro¬ 
curement  (TPP)  process.  New  Deputy 
Secretary  of  Defense  Packar^  issued  DoD 
Itirective  5000.1,  "Acquisition  of  Major 
Systems"  in  1971  to  establish  new  poficy 
with  emphasis  on  competent  people,  rational 
priorities,  clearly  defined  responsibilities  and 
adequate  authority  for  decentralized  acquisi¬ 
tion  of  approved  defense  systems.  Another 
new  Deputy  Secretary,  Carlucci,  instituted 
the  DoD  Acquisition  Improvement  Program 
in  1981.  Other  new  Defense  Secretaries  and 
D^uties,  and  new  top  policy  makers  in  the 
military  services  since  1947,  have  also  made 
significant  changes  in  policy,  oigar^tion- 
structure  and  processes  to  improve 
management  of  the  DSAM  process. 

In  1985,  President  Ronald  Reagan  issued 
Executive  C>rder  12526  to  establish  the  Presi¬ 
dent's  Blue  Ribbon  Conunission  on  Defense 
Management.  It  was  tasked  to  study  defense 
management  policies  and  procedures,  focus¬ 
ing  on  seven  particulars,  the  first  of  which 
was  "Review  the  adequacy  of  the  defense 
acquisition  process.”  '^e  Commission, 
ch^d  by  David  Packard,  and  known  as  the 
Packard  Commission,  submitted  its  final 
report,  A  Quest  for  Excellence:  Final  Report 
to  the  President,  on  June  30,  1986.^  Perhaps 
tiie  most  broad  based  anal^es  of  the  defense 
acquisition  process  ever,  it  included  recom¬ 
mendations  on  three  mqjor  management 
environments,  in  which  the  acquisition  proc¬ 
ess  operates  and  witii  which  it  interacts,  and 
"A  Formula  for  Action,"^  beginning  with  the 
following  quotation,  and  comprising  nine 
major  recommendations  on  organization  and 
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procedures  of  the  acquisition  process  itself. 

We  believe  it  is  possible  to  cut  this  cycle 
(ten-to-fifteen-year  acquisition  cycle)  in 
half.  This  will  require  radical  reform  of 
acquisition  organization  and  procedures. 

It  will  require  concerted  action  by  the 
Executive  Branch  and  Congress,  and  the 
full  support  of  the  defense  industry. 
Specifically,  we  recommend  that  the 
Administration  and  Congress  join  forces 
to  implement  the  following  changes  in  the 
acquisition  system. _ 

Coincident  with  the  highly-visible  increas¬ 
ing  need  to  improve  management  of  systems 
acquisition,  there  has  also  been  a  long¬ 
standing  less-recognized  need — to  increase 
productivity  substantially  in  the  DSAM 
process.  Seeking  insights  for  improvements 
that  would  significant^  increase  productivity 
of  the  overSl  DSAM  process,  we  have 
observed  and  an^zed  the  above  and  other 
major  acquisition  improvement  efforts. 

Most  have  recommended  specific  changes 
in  organizational  structures,  policies,  proc¬ 
esses  and  procedures  to  improve  manage¬ 
ment  of  systems  acquisition.  However,  most 
past  improvements  based  on  these  recom¬ 
mendations  have  not  increased  productivity 
in  the  DSAM  process  substantially.^  The 
Packard  Commission  goal  of  cutting  the 
acquisition  cycle  in  half  is  one  of  several 
measures  of  increasing  productivity  substan¬ 
tially,  but  their  record  on  achieving  this  goal 
is  still  out  since  all  of  their  recommendations 
have  not  yet  been  fully  implemented. 

The  need  to  increase  productivity  sub¬ 
stantially  in  DSAM  processes  is  even  more 
critical  today.  The  Soviet  nuclear  threat  is 
gone,  but  the  number  of  nations  with  nuclear 
capability  is  growing.  We  live  in  an  increas¬ 
ing  unstable  world  whose  threats  have  yet 
to  be  identified.  Yet,  the  DoD  is  downsizing, 
and  resources  for  new  defense  tystems  are 
being  significantly  reduced.  Witii  growing 
emphasis  on  deficit  reduction  and  increased 
domestic  spending,  there  is  pressure  for  even 
greater  reduction  in  defense  spending.  As  a 
result,  setting  acquisition  priorities  is  shifting 
emphasis  fi'om  new  system  acquisitions  to 
fielded  system  modifications  and  upgrades, 
or  other  less  costly  ways  to  meet  any,  yet 
undefined,  threat  to  our  national  security. 

We  face  a  critical  national  security 


problem:  How  to  get  more  defense  output 
from  less  input?  TTie  only  way  to  get  more 
usefiil  and  quality  defense  system  output, 
from  shrinking  defense  resources  input,  is  to 
increase  productivity  substantially  in  the 
DSAM  process.  Increasing  productivity 
substanti^  has  been  the  objective  of  our 
research,  begitming  in  the  mid-seventies,® 
and  is  the  objective  of  this  research  paper. 

The  Packard  Commission's  A  Quest  for 
Excellence,  like  earlier  acquisition  improve¬ 
ment  efforts,  also  recommended  specific 
changes  in  acquisition  organization  and  pro¬ 
cedures.  However,  the  Conunissioris 
"Formula  for  Action"  was  more  comprehen¬ 
sive,  including  actions  by  the  Congress  and 
industry  as  well  as  by  DoD  and  the  Executive 
Branch.  More  important,  Chairman  Packard 
provided  significant  new  insights  for  achiev¬ 
ing  excellence  in  acquisition  which  he  stated 
in  his  Foreword  to  the  report.  Following  are 
two  excerpts  from  lus  Foreword.’ 

In  any  laige  organization,  policies  must  be 
executed  throi^  discrete  structures.  In 
the  large,  complex  enterprise  of  national 
defense,  this  requires  that  we  cultivate 
resilient  centers  of  management 
excellence  dedicated  to  advancing  DoD's 
overall  goals  and  objectives. _ 

Excellence  in  defense  management  will 
not  and  can  not  emerge  by  legislation  or 
directive.  Excellence  requires  the 
opposite — responsibility  and  authority 
placed  firmly  in  the  hands  of  those  at  the 
working  level,  who  have  knowledge  and 
enthusiasm  for  the  tasks  at  hand.  To 
accomplish  this,  ways  must  be  found  to 
restore  a  sense  of  shared  purpose  and 
mutual  confidence  among  Congress, 
DoD,  and  industry.  Each  must  forsi^e  its 
current  ways  of  doing  business  in  favor  of 
a  renewed  quest  for  excellence. 
(underlines  for  emphasis) _ 

We  believe,  and  wUl  show  how  his  in¬ 
sights  can  enable  management  improvements 
that  can  increase  productivity  of  the  DSAM 
process  substantially. 

Purpose  and  Scope.  The  primary  purpose 
of  this  paper  is  to  summarize  or  identify 
enough  of  a  nearly  complete  set  of  concepts 
(space  does  not  permit  otherwise)  needed  to 
understand  the  critical  need  for,  and  large 
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payofl'  from  applying  new  IMT  to  implement 
the  set  of  concepts.  This  can  provide 
productive  .sj  stems  and  aids  for  professionals 
in  DS.WI/TDSM  centers  of  management 
excellence  throughout  government,  industry 
and  academia. 

We  have  three  objectives.  .-Ml  require 
DoD  action  to  implement. 

•  Begin  now  to  develop  a  two-level 
DSAM  K&I  Locator  System  -  to  increas¬ 
ingly  enable  timely  selective  access,  by  any 
DSAiVl  professional,  to  DSAM  K&I  when 
the  professional  needs  relevant  K&I  in  a  job 
or  for  a  task  at  hand  . 

•  Begin  now  to  develop  an  Integrated 
DS-\M  Research  &  Information  (R&I)  Proc¬ 
ess  System-  to  speed  filling  identified  voids, 
and  to  maintain  the  relevancy,  currency  and 
accessibility  of  the  body  of  DS.\M  K&I  that 
professionals  need  to  access.^ 

•  Plan  now  to  build  on  these  evolving 
DS.AM  K&I  l.ocator  and  R&I  Process  Sys¬ 
tems.  and  to  expand  them  to  IDSM  K&I 
Ix)cator  and  R&I  Process  Systems,  and  thus 
increase  productivity  even  more  in  an  II3SM 
process  that  covers  every  defense  .system's 
entire  life  cycle,  ^ 

Together,  these  will  increase  the  produc¬ 
tivity  of  all  DS.AM  professionals,  and  thus, 
increase  productivity  substantially  in  the 
DSAM  process;  and  ultimately,  increase 
productivity  even  more  throughout  a  seam¬ 
lessly  integrated  IDSM  process — for  acquir¬ 
ing  and  fielding  defense  systems,  maintaining 
and  upgrading  them  throughout  operational 
employment,  and  finally  disposing  of  them  in 
an  environmentally  suitable  matmer. 

Given  the  domain  of  the  DSM  life-cycle, 
the  objective  of  integrated  defense  systems 
management  (i.e.,  the  IDSM  process)  for 
any  system  should  be;  to  not  only  acquire 
and  field  a  QL,\I-ITY  hardware  system 
(operationally  effective  and  suitable  for 
intended  use''^  at  affordable  cost),  but  also  to 
maintain  this  QUAIiTV  in  the  integrated 
"total"  man-machine  system  throughout  its 
operational  employment,  and  finally,  to 
assure  its  environmentally  suitable  disposal. 

This  IDSM  objective  can  only  be  achieved 
by  seamlessly  integrating  the  present  different 
stages  of  defense  systems  management  (thus 
eliminating  costly  between-stage  transitions) 
throughout  the  life  cycle  of  every  system. 


Three  stages  that  must  be  seamlessly 
integrated  are; 

•  DSAM,  .Acquisition  Management 
through  each  system's  initial  deployment. 

•  (^ESM,  Support  .Management  through 
each  system's  operational  employment. 

•  DSDM  Disposal  Management  in  an 
environmentally  suitable  manner. 

Achieving  this  IDSM  objective  will  re¬ 
quire  networks  of  IDSM  Process  Systems; 
and  to  support  them,  an  IDSM  K&I  Locator 
System.  But  first,  we  must  focus  on  the 
underlying  concepts  and  the  conceptual 
framework  for  achieving  the  initial  objec¬ 
tive — to  develop  a  DS.AM  K&I  Locator 
System  that  will  improve  K&I  support  of 
DSAM  Process  Systems.  Then,  we  can 
build  upon  and  extend  this  conceptual 
framework.  and  increase  productivity 
substantially  throughout  (he  EDSM  process. 
We  use  the  acronym  DS.AMIDSM  when 
discussing  concepts  applicable  to  both 
processes. 

Since  all  of  these  concepts  cannot  be  dis¬ 
cussed  completely  in  this  short  paper,  num¬ 
bered  references  in  the  Endnotes  provide 
additional  detail  about  both  the  concepts  we 
summarize  or  identify,  and  those  omitted 
entirely  due  to  space  limitations.  Some 
earlier  concepts  have  been  modified  by  later 
developments  (e.g.,  by  tremendous  advances 
in  IMT  since  we  began  our  research  in  the 
mid-seventies).  Thus,  a  current  concept 
based  on  existing  IMT  may  differ  somewhat 
firom  an  earlier  version  that  was  based  on  our 
projections  for  emerging  technologies. 

Underlying  Concepts 

In  this  section,  we  summarize  or  identify 
several  underlying  concepts  which  we  believe 
win  help  understand  the  DSM  (DS.AM  and 
IDSM)  Knowledge  Systems  concepts 
outlined  in  the  next  section,  and  the  critical 
need  to  start  now  to  implement  the  concepts. 

Crucial  Consequences.  No  one  person 
or  organization  can  have  e.xperiential  knowl¬ 
edge  of  all  aspects  of  the  immense,  extremely 
complex  and  dynamic  DS.AM  proc¬ 
ess-  larger  and  more  complex  than  any 
management  process  in  the  business  world. 
Each  DSAM  professional,  no  matter  how 
expert  in  his  or  her  particular  DS.AM  field, 
function  and  subprocess,  suffers  from  two 


453 


crucial  consequences  of  the  inherent  large 
scope,  extreme  complexity  and 
"interactiveness"  of  the  DSAM  process,  and 
will  suffer  more  in  the  future  larger  IDSM 
process.  The  crucial  consequences  are: 

•  False  perceptions  of  tfie  whole  DSAM 
or  EDSM  process,  resulting  from  experience 
in  only  a  few  DSAM  or  IDSM  subprocesses. 
The  kinds  of  perceptions  are  important 
because  perceptions,  “  not  reality,  trigger 
change  and  guide  DSAM  decisions  and 
action. 

•  Misunderstood  communications,  among 
DSAM  professionals  from  different  fields, 
functions  and  with  different  working  jargon, 
which  result  from  lack  of  a  common  DSAM 
language  to  assure  cross-field  and  cross¬ 
function  communications  with  understanding. 

Both  of  these  consequences  are 
exacerbated  by  high  turnover  of  professionals 
in  key  jobs,  due  to  promotions  and  career 
advancement  moves,  and  particularly,  by 
concentrated  high  turnover  due  to  changes  in 
Administrations  every  4  or  8  years.  New 
Administrations  bring  new  teams  of  policy 
makers  with  new  national  goals  they  promise 
the  electorate  to  emphasize.  They  also  bring 
different  perceptions  of  the  DSAM  process 
and  its  importance  to  national  security. 

The  combined  results  of  these  inherent 
consequences  and  exacerbating  turnovers,  are 
wasteful  zigs  and  zags  in  major  DSAM 
policy,  and  costly,  schedule-delaying  reworks 
in  programs,  system  designs,  production 
processes,  etc.  in  the  DSAM  process. 

Considering  all  of  the  above,  our  an^sis 
indicates  that  developiitg  and  maintaining 
truer  perceptions  of  the  whole  DSAM  proc¬ 
ess,  and  also  developing  a  more  common 
DSAM  language,  are  prerequisites  for  suc¬ 
cess  in  future  efforts  to  increase  productivity 
substaniially  in  defense  acquisitions.*^ 

Prerequisites  for  DSAM  Excellence. 
Given  the  truer  perceptions  and  common 
language  prerequisites,  we  begin  with,  and 
will  continue  to  build  upon.  Chairman 
Packard's  discerning  insights,  quoted  above. 
These  insights  concern  defense  management 
in  general  and  particularly  the  DSAM 
process.  They  will  be  equally  applicable  in 
the  future  ISDM  process. 

•  A  prerequisite  requirement  for 
excellence  in  management  of  the  large, 


complex  enterprise  of  national  defense  is  that 
DSAM/IDSM  policy-makers  must  cultivate 
centers  of  management  excellence 
dedicated  to  advancing  DoD's  overall  goals 
and  objectives. 

•  Excellence  in  management  of  the 
DSAM/IDSM  process  requires  that  those  at 
the  working  level  have  firm  responsibility'  and 
authority,  and  also  knowledge,  and 
enthusiasm  for  the  tasks  at  hand. 

The  first  two  requirements  for  excellence 
in  management  -responsibility  and  authority 
at  the  working  level — can  be  met  through 
specific  organization  structure  and  policy 
recommendations.  Reorganizing  structures 
and  changing  policy  have  been  the  traditional 
focus  of  past  efforts  to  improve  the  DSAM 
process.  Also,  staff  selection  and  motivation 
have  been  the  traditional  means  for  assuring 
the  enthusiasm.  But  these  are  not  sufficient 
for  excellence!  They  do  not  meet  the  re¬ 
quirement  of  "knowle<tee  for  tasks  at  hand." 

Packard's  perceptive  "requirements  for 
excellence"  insights  raise  "what  is  necessary" 
and  "how"  questions.  For  example: 

1.  What  is  necessary  for  productive 
cultivation  and  maintenance  of  (DSAM, 
ultimately  IDSM)  centers  of  management 
excellence  which: 

•  can  best  assure  timely,  coordinated, 
effective  execution  of  centralized  DoD 
systems  management  policies  through 
necessarily  discrete  structures, 

•  operate  in  ever-changing  political,  eco¬ 
nomic  and  international  environments  (e.g., 
post-Soviet  environment)  that  may  necessi¬ 
tate  changes  in  the  centralized  DoD  policies, 
and 

•  can  continually  coordinate  their  respec¬ 
tive  policy  execution  efforts  with  other 
centers,  in  order  to  contribute  effectively 
toward  achieving  the  current  national  defense 
and  DoD  goals  and  objectives. 

2.  Given  successful  cultivation  and  main¬ 
tenance  of  DSAM  centers  of  management 
excellence  (hereafter  often  called  centers); 

•  How  best  can  the  most  relevant  DSAM 
K&I  centers  be  quickly  located? 

•  How  best  assure  these  K&I  centers  can 
be  selectively  accessed  by  any  DSAM  profes¬ 
sional  to  get  timely  DSAM  K&I  relevant  to  a 
task  at  hand? 

3.  Concerning  Packard's  requirement  of 
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knowledge  for  tasks  at  hand: 

•  How  best  assure  that  all  K&I  needed 
for  tasks  at  hand  are  available? 

•  When  not  available,  how  to  expedite  the 
needed  K&I  to  the  professional  who 
identified  it,  and  also  assure  it  is  added  to  fill 
the  void  in  the  body  of  K&I,  and  maintain  it 
for  future  timely  access? 

Before  answering  these  questions,  we 
need  to  consider  other  building  block  con¬ 
cepts  developed  in  our  research  that  should 
help,  both  in  understanding  the  DSAM  and 
IDSM  knowlet^e  system  concepts  generated 
by  answers  to  these  questions,  and  in  imple¬ 
menting  the  concepts.  Then  we  will  continue 
our  examination  of  means  for  meeting  these 
Prerequisites  for  Excellence. 

Productivity  Elements  Framework. 

U'e  developed  a  DS  AM-process  productivity 
eltments  framework  concept*^  as  an  analyti¬ 
cal  tool.  Its  purpose  is  to  help  systematically 
identify  and  consider  all  productivity  ele¬ 
ments  that  can  influence  the  DSAM  process 
output,  in  order  to  concentrate  on  those  ele¬ 
ments  that  promise  largest  increases  in  pro¬ 
ductivity.  Our  current  productivity  element 
framework’^  includes  three  traditional  organ¬ 
izational  elements  objectives,  processes  and 
structures  and  two  types  of  DSAM  process 
inputs  (1)  tangible  resources,  and  (2)  intan¬ 
gible  job-  and  task-related  DS.\M  K&l  re¬ 
sources.  llie  intangible  K&l  resources, 
great  contributors  to  increased  productivity, 
have  usually  been  overlooked  in  past  im¬ 
provement  {TTograms.  Two  more  elements, 
the  environment  and  culture  of  the  acquisi¬ 
tion  process,  were  added  because  of  the 
Packard  ('ommission  report  on  how  much 
the  current  environments  ol  defense  acquisi¬ 
tion  and  the  present  defense  management 
culture,  were  and  still  are,  adversely  affecting 
the  acquisition  prote.s.s. 

Do!)  Proiliictivity  Initiatives.  We  u.scd 
the  productivity  tramevvork  in  1989  to  iden- 
tifv  new  (jpporlunilies  to  inereav  productiv- 
it\  ol  DS.AM  professionals  ''  Ba.sed  on 
results  Ironi  the  19X8  review  ol  past  eflorts 
lo  improve  the  acquisition  process,'*  we 
concluded  that  l)ol)  initiatives  in  three  inter¬ 
dependent  DS.AM  areas  could  expedite  im- 
piovements  in  the  DS.AM  process.  The  three 
Dol )  initiatives  are; 

1.  (’ontinue  increasing  the  integration  of 


major  DSAM  processes. 

2.  Manage  constructive  change  of 
defense  acquisition  environments,  and 
motivate  improvements  in  the  present 
defense  management  culture. 

3.  Provide  DSAM  knowledge  system  aids 
that  can  increase  productivity  of  DSAM 
professionals. 

DSAM  Knowledge  System  Concept. 
Figure  1  is  a  simplified  model  of  an  inte¬ 
grated  Research  &  Information  (R&I) 
DSAM  process.'^  It  represents  a  partial 
answer  to  the  last  two  questions  above  under 
Prerequisites  for  DSAM  Excellence.  This 
process  is  required  to  support  the  cultivation, 
maintenance  and  ongoing  operations  of  all 
DSAM  centers  in  the  defense  acquisition 
community.  The  databases  represented  by 
some  of  the  boxes  may  be  distributed  and  in 
different  locations.  The  numbered  arrows 
indicate  four  processes  integrated  into  the 
R&I  process.  Arrows  outgoing  from  the 
DSAM  professional,  who  needs  relevant 
DSAM  (K&I),  enter  four  boxes.  These 
boxes  represent  four  categories  of  DSAM 
R&I  databases,  which  may  be  components  of 
larger  databases  in  organizations  located 
throughout  the  DSAM  community.  These 
databases  are  important  elements  of  a  DSAM 
K&I  Locator  System  which  we  discuss  later. 

Any  professional  could  use  this  integrated 
process  to  get  particular  DSAM  K&I  for  a 
task  at  hand.  First,  the  professional  would 
use  DSAM  K&I  Ix)cator  System  to  query  ( 1 ) 
DSAM  Documents  databases  for  K&I  which 
have  been  documented,  and  also  (2)  DS.AM 
E.xpertise  databases  lo  locate  experts, 
individuals  or  organizatioas,  available  for 
consultation.  If  both  queries  yield  nothing 
aseful,  the  professional  would  check  (3) 
Ongoing  DS.AM  Research  Projects  databases 
of  DSAM  research  organizations  to  identify 
piincipal  investigators  who  have  the  requited, 
but  as  yet  unpublished  knowledge.  Success 
in  any  of  these  queries  would  close  the  loop 
by  expediting  the  sought  K&l  to  the 
professional,  enabling  him  or  her  to  complete 
the  task  at  hand. 

If  no  queiy  of  these  three  categoncs  of 
databases  produces  the  required  DS.AM 
K&I.  the  professional  would  register  the 
sought  item  ot  K&l  (as  a  void  to  be  tilled  in 
the  body  of  DSAM  K&l,  and  iheicloic,  a  re¬ 
search  requirement)  in  the  \  alidated  DS.AM 


Fig  1.  Integrated  Research  and  Information  (R&I)  Process 


Issues/Problems  (VDIP)  data¬ 
base.  The  K&I  item  needing 
research  would  be  entered  into 
the  Projects  Being  Evaluated 
(PBE)  ffle  of  the  VDIP  data¬ 
base,  for  evaluation  by  compe¬ 
tent  authorities.  If  they  evaluate 
it  as  a  sound  requirement  for 
new  DSAM  K&I  and  give  it  a 
research  priority,  the  re¬ 
quirement  would  be  transferred 
to  the  Validated  Candidate  Pro¬ 
jects  (VCP)  file  of  the  VDIP 
database  as  a  validated  project 
for  DSAM  research  with  an 
assigned  priority.  One 

centralized  DoD  VDIP  database 
would  serve  all  DSAM  research 
organizations  in  the  DSAM 
acquisition  research  community. 

If  the  professional's  requirement  receives  a 
high  priority,  a  DSAM  research  organization 
which  has  an  interest  in  the  validated  candi¬ 
date,  expertise  in  the  DSAM  area,  and  the 
necessary  resources,  would  select  and  trans¬ 
fer  the  candidate  project  to  its  own  Oi^oing 
Research  Projects  database,  and  begin 
research.  After  completing  the  project,  the 
principal  investigator  would:  (1)  provide  the 
new  K&I  directly  to  the  initiator  professional 
who  needs  it  to  complete  a  task,  (2)  publish 
the  research  results  to  fill  a  void  in  the  body 
of  DSAM  knowledge,  and  (3)  finally  add  the 
published  report  to  a  DSAM  Documents 
database  for  future  retrieval  when  needed  by 
another  professional. 

Implementing  this  integrated  R&I  process, 
a  DSAM  Knowledge  System  concept,  will 
best  assure  that  job-  and  task-relevant  DSAM 
K&I  are  available  for  access  by  any  profes¬ 
sional,  and  will  expedite  development  of  new 
K&I  to  fill  voids  and  maintain  a  useful  body 
of  DSAM  K&I  for  access  when  needed. 

Prerequisites  for  Excellence  Contin¬ 
ued.  Neither  Chairman  Packard,  nor  the 
1986  Commission  report,  considered  how  or 
where  to  cultivate,  or  how  to  maintain, 
centers  of  management  excellence.  Nor  did 
they  coasider  how  to  assure  that  those  at  the 
"working  level"  could  always  get  timely 
knowledge  tor  each  task  at  hand. 

Initiall>’  we  sought  to  understand  the  new 
aspects  of  Packard's  requirements  for  excel¬ 


lence  by  asking  the  three  questions  stated 
above  under  Prerequisites  for  Excellence. 

Then,  seeking  answers  to  the  first  "what  is 
necessary"  question,  we  focused  on  three 
"ted  DSAM  and  IDSM  issues,  seeking 
p  tr-t''  r.:  DSAM  insights  with  early  payoff, 
wl  ;ch  )uld  be  extended  within  an  ultimate 
IDSM  framework:’® 

•  What  management  knowledge  (and 
information)  is  required  for  excellence  to 
emerge,  and  what  conditions  are  necessary 
for  excellence  to  flourish? 

•  What  is  the  domain  of  Integrated 
Defense  Systems  Management  (IDSM)  when 
systems  management  is  extended  beyond 
DSAM  to  include  all  of  a  defense  system's 
life  cycle? 

•  What  is  necessary'  to  integrate  the  three 
now  separate  DSM  stages  into  an  Integrated- 
DSM  (IDSM)  process  over  each  defense 
system's  life  cycle? 

Knowledge  Required  by  Centers.  To 

answer  the  first  question,  we  focused  on  how 
to  meet  ("hairman  Packard’s  requirements  for 
centers  for  management  excellence  that 
cannot  be  filled  by  cunent  organization  and 
management  practice.  There  are  two; 

•  How  to  develop  and  maintain 
specialized  know-how  for  application  by  any 
DSAM  professional  when  needed. 

•  How  to  facilitate  each  professional’s 
exploration  of  new  and  better  w^  s  to  get  his 
or  her  job  done. 

Ihe  first  can  be  met  by  providing  each 
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center  with  systems  and  aids  for  timely, 
selective,  on-demand,  access,  by  any  mem¬ 
ber,  to  relevant  sources  of  any  needed 
specialized  know-how.  In  addition,  the  aids 
must  support  exploration  of  extern^  somces 
for  new  (to  the  individual)  and  better  ways  to 
get  their  job  done.  How  to  develop 
knowledge  systems  and  aids  which  can  do 
both  will  be  discussed  later  after  some  con¬ 
cepts  that  show  what  is  involved  are  visited. 

The  answer  to  "where  to  cultivate"  centers 
of  management  excellence  in  order  to  pro¬ 
vide  all  DSAM  professionals  access  to  these 
sources  is  that  they  must  be  cultivated  and 
maintained  at  every  source  of  DSAM  K&I, 
and  every  point  of  DSAM  decision  and 
action,  throughout  the  DSAM  community  (in 
government,  industry 
and  academia),  and 
intercoimected  by 
knowledge  systems. 

How  to  do  this  will 
also  be  discussed. 


DSM  Concept.  The 
Packard  Commission 
showed  little  concern 
about  program  man¬ 
agement  of  defense 
systems  during  the  rest 
of  a  system's  Itfe  cycle. 

Our  '  1990-1991 

analysis  of  the  issues 
listed  above  under 
Prerequisites  for 
Excellence,  indicated 
that  extending  the 
domain  of  defense 
systems  management 
throughout  the  entire 
life  of  each  defense  system  would  give 
significant  payoff-  -in  er^anced  operational 
suitability  and  operational  effectiveness  of 
each  defense  system,  in  dollars  saved,  in 
more  productive  integrated  defense  systems 
management  practice,  and  in  improved 
education. 

We  used  "systems  thinking"  to  defme  and 
discuss  the  Defense  Systems  Mana^jement 
(DSM)  domain  that  included  and  extended 
beyond  DSAM  to  cover  each  system's  life 
cycle.  We  discussed  the  boundaries  of  the 
DSM  domain,  and  how  they  spanned  both  a 
great  managerial  hierarchy  and  a  time  line 


(DSM  time  line).  The  managerial  hierarchy 
ranges  from  establishing  all  national  defense 
policies  that  involve  defense  systems,  down 
through  numerous  oiganizational  hierarchies 
to  the  management  level(s)  where  these 
policies  are  finally  executed. 

The  upper  part  of  Figure  2  portrays  our 
new  concept  of  a  Defense  Systems  Man¬ 
agement  (DSM)  view  of  a  defense  system's 
life  cycle,  with: 

•  Three  consecutive  stages:  DSAM, 
OESM,  and  DSDM. 

•  Concepts  of  several  categories  of  DSM 
K&I  Process  Systems  that  we  believe  are 
required  to  support  the  centers  in  each  stage. 
There  are  many  more  K&I  process  systems. 
Many  alreacfy  exist,  others  should  be  de¬ 


veloped,  and  all  need  to  be  mtegrated.  (See 
K&I  Process  Systems  below. ) 

•  Some  kinds  of  DSM  K&I  that  a  DSM 
K&l  Locator  System  could  provide. 

The  boundaries  behveen  DSM  stage 
processes  are  the  result  of  some  significant 
differences  in  the  "culture"  and  objectives  of 
the  professionals  working  in  each  process. 
Disconnects  and  misunderstandings  in  com¬ 
munications,  and  discontinuities  in  manage¬ 
ment  processes,  result  when  professionals 
working  in  a  particular  process  don't  have  a 
completely  common  language.  Integrated 
DSM  will  be  difficult,  if  not  impossible. 


DEFENSE  SYSTEMS  MANAGEMENT  (DSM) 

over  the 

Life  Cycle  of  Each  Defense  System 

Acquisition 

DSAM 

Stage 

Operational  Employment  Support 
OESM 

Stage 

Disposition 

DSDM 

Stage 

1  DSM  K&I  (knowledge  &  information)  Process  Systems  | 

DSAM  K&I 

DT&E  K&I - 

M-MIM  K&I-- 
L&ISMK&I--- 

OESM  K&I 

-  OT&E  K&l 

DSDM  K&l 

-  -L&ISM  (ILSM&ISM)  K&l - 

- ILSM  K&I 

1  Kinds  of  K&I  Provided  by  DSM  K&I  Locator  Systems  | 

Milestones 

0-IV 

Phases  1-lV 
FMPT&S  Prog 

L  &  IS  Plaiming 

Operational  Test  &  Evaluation 
Man-Machine  Integration 

Integ  Log  Spt  &  Irifrastructure  Support 

Foreign  Military 
Sales 

Recycle/ Salvage 
EnvuonmentaT 
clean  up 

INTEGRATED  DEFENSE  SYSTEMS  MANAGEMENT  (IDSM) 

IDSM  over  the  Life  Cycle  of  each  Defense  System  can  be  achieved 
by  interconnecting  all  Centers  of  Management  Excellence,  and  all 

DSM  Knowledge  Systems,  with  a  Standardized  Common  IDSM  Language 

Fig.  2  DSM  (Defense  Systems  Management)  &  IDSM 
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without  common  concepts  and  a  common 
working  language. 

IDSM  Concept.  The  last  part  of  Figure  2 
portrays  the  Integrated-DSM  (IDSM)  view 
of  a  defense  sy  stem's  life  cycle.  Implement¬ 
ing  this  integrated  view  will  involve  evolu¬ 
tionary  integration  of  the  standardized  com¬ 
mon  language  (working  terminology)  of  each 
former  DSM  stage  into  one  standardized 
common  IDSM  language.  This  IDSM  lan¬ 
guage  would  enable  development  of  DSM 
knowledge  systems,  using  IMT.  to  intercon¬ 
nect  all  centers  and  all  IDSM  Knowledge 
Systems  throughout  the  IDSM  community, 
thus  enabling  timely  access  to  relevant  IDSM 
K&I.  and  communication  of  these  K&I 
throughout  the  IDSM  community  (i.e.,  clear 
and  timeh’  K&I  feedback  and  "feedforward" 
across  former  DSM  stage  boundaries). 

This  IDSM  concept  requires  and  depends 
on  an  "mlormational"  integration  of  all  K&I 
required  for  management  tasks  in  any  former 
DSM  stage.  No  other  practical  alternative  is 
available.  Changes  in  organizational  structure 
may  correct  some  authority-responsibility 
deficiencies  (while  creating  offers)  but  they 
do  not  solve  the  problems  of  assuring  timely 
access  by  IDSM  professionals  in  any  part  of 
the  IDS\I  life  cycle,  to  K&I  that  are  relevant 
lor  ta.sks  at  hand. 

In  the  complex  bureaucracy  concerned 
with  national  defense,  IDSM  cannot  be 
achieved  organizationally.  But,  IDSM  can  be 
achieved  operationally  -by  applying  new 
IDSM  Knowledge  Systems  concepts  and 
1  )iivid  Packard's  concept  of  centers  of  man¬ 
agement  excellence.  to  implement 
' ' informational"  integration  of  DSM  K&I. 

IDS.M  can  be  achieved  by  providing 
professionals  in  each  center  throughout  the 
IDSM  community,  timely  access  to  IDSM 
K&l  that  each  requires  for  five  main 
purposes: 

•  fo  continually  know  and  have  a  com¬ 
mon  understanding  ol  the  goals  and  objec- 
ti\  es  ol  DoD's  centralized  policies. 

•  lo  clearly  coordinate  each  center’s 
respective  etfoiis  in  its  area  of  respoasibiU- 
lies.  toward  coordinated  e.xecution  of  the 
commonly  understood  centralized  defense 
systems  management  policies,  in  order  for  all 
to  contribute  productively  to  achieving  DoD's 
goals  and  objectives. 


•  To  develop  and  maintain  job-  and  task- 
related  specialized  know-how. 

•  To  explore  new  and  better  ways  for 
getting  his  or  her  job  done. 

•  To  perform  with  excellence  any  man¬ 
agement  tasks  at  hand. 

We  should  note  that  professionals  in  all 
centers  throughout  the  present  DSAM, 
OESM,  and  DSDM  stages  also  need  K&I  for 
the  same  five  purposes — to  interact  with  their 
knowledge  systems  which  help  them  per¬ 
form  their  respective  functional  processes. 

DSAM  Knowledge  Systems 

Having  used  Figure  2  to  help  explain  the 
ultimate  potential  of  IDSM,  we  turn  now  to 
our  first  objective:  to  urge  DoD  action  to 
begin  developing  a  DSAM  K&I  Ix)cator  Sys¬ 
tem  that  can  locate  and  access  the  kinds  of 
K&I  portrayed  in  the  left  DSAM  part  of 
Figure  2.  But  first  a  quick  look  at  the  other 
kind  of  knowledge  system,  process  systems, 
and  their  relationship  to  the  locator  system. 

DSAM  K&l  Process  Systems.  There 
are  many  process  systems  but  only  one 
locator  system.  Space  does  not  permit  sum¬ 
marization  of  DSAM  K&l  Process  Systems 
except  to  note  that  these  systems  largely 
determine  the  kinds  of  K&I  that  DSAM 
professionals  must  locate  and  selectively 
access.  The  professionals  require  K&I 
mainly  to  interact  with — provide  K&I  inputs 
to  and  receive  K&I  from— DS, AM  K&I 
Process  Systems,  and  to  operate  or  design, 
develop  and  improve  [jrocess  systems  needed 
to  help  the  professionals  achieve  the 
objectives  of  their  DS.-kM  subprocesses. 

The  Integrated  DS.AM  R&I  Process 
System  for  maintaining  the  body  of  DSAM 
K&I  is  an  example  of  K&I  process  systems. 
Three  other  examples  dealing  with  systems 
management  processes  are  identified  in 
Figure  2. 

•  DT&E  (Development  Test  &  Evalu¬ 
ation)  K&I  lYoccss  Systcm(s). 

•  M-MIM  (Man-.Machine  Integration 
Management)  K&I  Process  System(s). 

•  L&ISM  (Logistics  and  Infrastructure 
Support  Management)  K&I  Process 
System(s). 

These  three  examples  are  based  on 
processes  documented  in  DoD  Instruction 
5000.2.  M-MIM  represents  our  process 
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system  concept  of  integrating  the  man 
(people)  processes — forces,  manpower, 
personnel  training,  human  factors,  safety, 
etc.  — together  in  man-machine  integration 
systems.  The  example  systems  also  generate 
and  transmit  K&l  as  weU  as  receive  K&I,  for 
management  of  specific  defense  system 
pro^ams,  or  for  l^er  level  management 
apphcable  to  many  defense  systems.  We 
believe  that  these  process  systems  should 
expedite  K&I  across  DSM  stage  boundaries 
(indicated  by  dashed  lines  in  Figure  2)  in 
order  to  support  progressive  integration  of 
the  now  separate  DSM  stage  processes, 
toward  the  ultimate  IDSM  process. 

DSAM  K&l  Locator  System.  Thus  far, 
we  have  looked  at  several  factors  that  affect 
the  kind  of  K&I  systems  and  aids  which  must 
be  developed  and  maintained  to  support  the 
cultivation,  emergence  and  maintenance  of 
DS,\M  centers  throughout  the  DSAM 
community.  The  review  indicates  that  the 
DSAM  K&I  systems  and  aids  must  be  able 
to  provide,  to  the  professionals  in  these 
DS,\M  centers,  on-demand,  timely,  selective 
access  to  DS  .AM  K&I  that  are  relevant  to  any 
of  the  five  main  purposes  identified  above 
under  the  EDSM  Concept. 

These  stringent  on-demand,  selective, 
relevant  K&I  access  requirements  for 
maintaining  centers  throughout  the  DSAM 
community  fly  in  the  face  of  harsh  reality. 
The  enormous  bo^  of  DSAM  K&I  is 
distributed  widely — in  many  collections: 

•  of  different  categories  of  DSAM  K&I 
(contracting,  systems  engineering,  etc.), 

•  with  many  different  formats  (reports, 
books,  articles,  electronic  data,  CD-ROM, 
human  expertise),  and 

•  assembled  to  support  many  different 
purposes  and  functions  in  DSAM  process. 

These  collections  are  located  in  thousands 
of  different  organized  DS.AM-area  K&I 
SOURCES  (e.g.,  in  special  libraries, 
repositories,  archives,  databases,  centers  of 
expertise,  information  analysis  centers). 

Considering  both  the  stringent  K&I  access 
requirements,  and  the  size  and  wide 
distribution  of  the  body  of  DSAM  K&I: 

•  flow  best  can  all  of  these  items  of 
DSAM  K&I  be  identified,  organized, 
cla.s.sified,  and  indexed,  for  selective,  on- 
demand,  access  and  retrieval,  by  DSAM 


professionals  in  centers  throughout  the 
DSAM  community? 

Our  answer  to  this  question  after  much  re¬ 
search  analysis  is; 

•  By  beginning  to  implement  a  two- 
level  DSAM  K&I  Locator  System,  with: 

•  a  macro-level  DoD  DSAM  Informa¬ 
tion  System.  Its  purpose  will  be  to  help 
DSAM  professionals  quickly  locate  known 
organized  SOURCES  of  DSAM-area  K&I 
that  likely  contain  K&I  which  are  relevant  to 
the  DSAM  professional's  purpose. 

•  thousands  of  micro-level  DSAM- 
area  Knowledge  Svsterrts — at  least  one  per 
SOURCE.  The  purpose  of  each  knowledge 
system,  after  the  DSAM  professional  selects 
a  likely  relevant  SOURCE,  will  be  to  provide 
timely,  selective,  on-demand,  access  to 
specific  K&I  in  that  SOURCE,  which  are 
relevant  to  the  professional's  immediate 
ptirpose. 

The  following  summaries  are  meant  to 
help  understand  the  two-level  Taxonomy- 
Glossary  standardized  structure  for  Informa¬ 
tion  and  Knowledge  Systems,  realize  the 
usefulness  and  key  roles  of  boft  levels,  and 
recognize  the  critical  need  for  DoD  to  act 
now  to  implement  the  DoD  DSAM-SIS  con- 
cept,  and  to  encourage  evolutionary  imple¬ 
mentation  of  the  DS  /\M-area-SKS  concept. 

DoD  DSAM  Information  System.  The 

upper  level  DoD  DSAM-SIS  (DoD  DSAM 
Structure  &  Information  System)  is  the  DoD 
DSAM  Information  System.  This  Irtforma- 
tion  System  is  structured  (its  DSAM-area  and 
location  data  organized,  classified,  and  in¬ 
dexed  for  retrieval)  by  a  macro-level  DoD 
DSAM  Taxonomy-Glossary  of  the  body  of 
DSAM  knowledge  (summarized  below). 
Implemented  top-down,  the  DoD  DSAM- 
SIS  can  serve  many  purposes.  Initialfy,  the 
DoD  DSAM  Information  System  part  of  the 
DS.AM-SIS  will  provide  any  DSi^l  profes¬ 
sional  the  name  and  location  of  all  known 
major  organized  SOLUCES  of  particular 
categories  of  DSAM-area  K&I. 

The  DoD  DSAM  Taxonomy-Glossary 
(the  macro-level  DS.AM  Structure  part  of  the 
DS.AM-SIS)  will  be  standard  throu^out  the 
DSAM  community,  and  thus,  will  intercon¬ 
nect  all  DSAM  k&I  Knowledge  Systems 
into  an  integrated  network  of  DSAM-IKS 
(Integrated  Knowledge  Systems)  that  DSAM 
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professionals  will  be  able  increasingly  to 
access  directly  to  get  needed  relevant  DSAM 
K&I. 

DSAM-area  Know^dge  Systems.  ITie 
lower  level  DSAM-area-SKS  (DSAM-area 
Structure  &  Knowledge  Systems)  comprise 
thousands  of  micro-level  DSAM-area 
Knowledge  Systems^ — one  DSAM-area 
Knowledge  System  for  each  major  organized 
SOURCE  that  is  included  in  the  DoD 
DS/WI  Infom\ation  System. 

Each  DSAM-area  Knowledge  System  will 
be  structured  by  its  respective  DSAM-area 
Taxonomy-Glossary  (or  any  other  indexing 
and  retrieval  system  that  each  SOLTR.CE  now 
has  until  it  develops  a  Taxonomy-Glossary) 
for  organizing,  classilying,  indexing,  storing, 
and  retrieving  (selectively  accessing)  specific 
items  of  its  DSAM-area  K&I. 

DSAM  Knowledge  Structure.  The 

structure  of  a  body  of  defense  systems  man¬ 
agement  knowledge  involves  relationships 
among  preferred  key  terms  of  the  language 
used  in  communicating  and  in  using  tlwt 
knowledge.  The  structure  is  not  initially 
unique,  may  be  somewhat  arbitrary,  and  is 
divided  into  a  macro  and  micro  level  because 
of  the  large  size,  wide  distribution  and  multi¬ 
discipline  nature  of  the  body  of  DSAM 
knowledge.  However,  the  .structures  (terms 
and  relationships)  adopted  for  both  levels  of 
DSAM  Taxonomy-Glossaries  must  be 
useful  used,  and  accepted  by  the  profes¬ 
sionals.  Only  then  will  the  Taxonomy-Glos¬ 
saries  be  useful,  accepted  and  used  by 
DSAM  professionals  for  assembling,  organ¬ 
izing,  classifying,  storing  indexing,  accessing, 
and  communicating  their  professional  knowl¬ 
edge  in  systems  man^ement  activities. 

To  be  most  useful,  and  both  clear  and 
unambiguous,  our  research  indicates  that  the 
structure  must  include  two  elements;  a 
DSAM  Taxonomy  and  an  integrated  DSAM 
Glossaiy  (with  defmition(s)  or  description  of 
each  term  in  the  Taxonomy).  llie  ^ossary 
serves  two  purposes:  to  assure  common 
understanding  and  use  of  each  term  in  the 
taxonomy,  and  to  promote  a  common 
DSAM  lan^ge  for  managing  defense 
system  acquisition,  A  DSAM  Taxonomy 
without  an  integrated  Glossary  implicitly 
assumes  a  completely  common  language 
(working  jargon)  among  all  profcs-sionals 


throughout  the  DSAM  community.  Not 
true,  hence  the  integrated  DSAM  Glossary. 

DSAM  Taxonomy-Glossary.  Webster 
defines  a  taxonomy  as  "The  systematic  dis¬ 
tinguishing  ordering  and  namii^  of  type 
groups  within  a  field."  The  taxonomy  of  a 
field's  knowledge  is  the  bootstrap  by  which 
professionals  in  any  field  inventory  their 
professional  knowledge,  identify  voick  to  be 
filled,  organize  research  to  fill  them,  achieve 
professionalism,  plan  integrated  operations, 
increase  productivity,  and  advance  their 
field’s  contribution  to  society.  Our  field  is  the 
body  of  DSAM  knowledge. 

The  DoD  DSAM  Taxonomy-Glossaiy 
that  is  developed  will  be  the  macro-level 
umbrella  Taxonomy  of  all  DSAM  areas  of 
this  whole  body  of  knowle(%e.  This  um¬ 
brella  DSAM  Taxonomy  will  connect  down 
to  each  micro-level  DSAM-area  Taxonomy 
as  it  is  developed,  thus  providing  a  complete, 
integrated,  hierarchical  DSAM  Taxonomy 
for  that  area  of  DSAM  knowledge. 

This  umbreUa  Taxonomy  will  interconnect 
all  of  its  named  DSAM  areas  to  show  reb- 
tionships,  and  so,  will  link  each  DSAM-area 
Knowle(^e  System  as  its  development  is 
completed,  into  the  integrated  DSAM-IKS 
network.  Ultimately,  when  all  micro-level 
DSAM-area  Taxonomy-Glossaries  are  de¬ 
veloped  (for  all  DSAM-area  Knowledge 
Systems),  and  connected  to  the  macro-level 
DoD  DSAM  Taxonomy-Glossary,  the 
combined  macro-level  and  micro-levels 
would  be  a  complete  DSAM  Ta.xonomy- 
Glossary  of  the  bo^  of  DSAM  knowledge. 

Recommended  DoD  Actions 

Under  DoD  Productivity  Initiatives  above, 
we  listed  three  general  initiatives  that  our 
1988-89  analysis  indicated  DoD  should 
pursue  to  expedite  improvements  in  the 
DSAM  process.  The  actions  recommended 
below  focus  primarily  on  the  third 
productivity  initiative  because  systems  and 
aids  that  can  increase  productivity  of  DSAM 
professioiuls,  promise  a  high-payofif  use  of 
DoD  resources.  They  focus  on  the  first  two 
objectives  stated  at  the  begiiuung  of  the 
paper,  because  development  of  the  DSAM 
K&I  Locator  and  DS,\M  R&I  Process 
Systems  must  get  underway  before  acting  on 
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the  third  objective. 

DoD  Actions  to  Start  Locator  System. 
Three  actions  are  necessary  to  start  d^elop- 
ing  the  macro-level  DoD  DSAM-SIS  and  its 
Information  System  which  will  provide  loca¬ 
tions  of  DSAM-area  SOURCES.  The  fourth 
will  support  evolutionary  development  of 
micro-level  DSAM-area  Knowle<^e  Systems 
to  increasing  provide  DSAM  professionals 
timely  selective  access  to  relevant  K&I,  after 
a  likefy  SOURCE  is  chosen.  The  actions  are: 

1.  Start  development  of  a  prototype 
macro-level  DoD  DSAM  Taxonomy  (first 
the  part  of  DSAM-SIS)  which  can  be  field 
tested  by  DSAM  professionals  and  improved 
in  structure  and  scope. 

2.  Simultaneously,  use  the  terms  in  the 
Taxonomy  to  begin  integrated  development 
of  the  DSAM  Glossary  to  assure  precision. 

3.  Start  developing  the  second  part  of  the 
DoD  DSAM-SIS— <i.e.,  the  DoD  DSAM 
Information  System).  As  the  DSAM-area 
structure  of  the  prototype  DoD  DSAM 
Taxonomy-Glossary  begins  to  emerge,  begin 
identifying  major  organized  SOURCES  of 
DSAM-area  K&I  (e.g.,  special  libraries,  re¬ 
positories,  databases,  centers  of  expertise). 
Use  the  DoD  DSAM  Taxonomy-Glossary  to 
classify,  index  and  enter  each  SOURCE  with 
its  data  into  the  growing  DoD  DSAM 
Information  System  database.  Also  use  the 
Taxonomy-Glossary  to  develop  a  "DoD 
Thesaurus  of  Sources  of  DSAM  Knowledge 
and  Information."  Then,  every  DSAM 
professional  can  use  the  Thesaurus  to  identify 
and  contact  those  SOURCES  which,  in  their 
respective  organized  DSAM-area  collections, 
most  likely  luve  DSAM  K&I  that  are  rele¬ 
vant  to  the  professional's  immediate  purpose. 

4.  Encourage  every  DSAM-area 
SOURCE  to  develop  a  micro-level  DSAM- 
area  Taxonomy-Glossary  for  classifying, 
indexing,  storing  and  retrieving  the  K&I  in 
its  DSAM  collection.  This  will  enable 
selective  access,  by  any  DSAM  professional, 
to  K&I  in  the  collection  that  are  relevant  to 
the  professional's  inunediate  purpose. 

DoD  Actions  to  Start  R&I  Process 
System.  Two  actions  are  necessary  to  start 
two  DoD  databases  which,  together  with  the 
DSAM  K&I  Locator  System,  will  comprise 
the  DSAM  R&I  Process  System  for  integrat¬ 


ing  the  DSAM  research  and  information 
processes: 

1.  Start  development  of  a  DoD  Validated 
DSAM  Issues/Problems  Database. 

2.  Start  development  of  a  DoD  Ongoing 
DSAM  Research  ^ojects  Database. 

ODier  DoD  Actions.  DoD  should  also 
begin  two  other  actions^^  that  support  the 
first  two  general  DoD  initiatives  cited  above. 
The  actions  are:  develop  an  integrated  hierar¬ 
chical  DoD  Acquisition  Strategy  to  help 
coordinate,  at  all  levels,  decentralized  execu¬ 
tion  of  defense  acquisition  policy  to  achieve 
DoD  acquisition  objectives;  and  conduct  a 
regular  Packard-Commission-like  review  of 
the  defense  acquisition  process  every  four 
years  to  get  broad  based  objective  informa¬ 
tion,  for  achievir^  constructive  change  in 
both  the  acquisition  environment  and  the 
acquisition  jvocess,  and  to  support  a  more 
orderfy  transition  to  new  Administration  DoD 
policy  makers.  Space  does  not  permit  sum¬ 
marizing  these  actions,  but  earlier  surtunaries 
are  ava^ble  in  the  Endnote  reference. 

SUMMARY 

Although  space  limitations  required 
omission  of  several  concepts,  we  believe  tiie 
concepts  that  were  summarized  or  identified 
should  achieve  our  purpose.  It  is  to  help 
understand  the  present  critical  need  for,  and 
large  payoff  fi:om  using  IMF  to  implement 
the  complete  set  of  concepts,  particulaily  the 
key  concepts  stated  earlier  in  tire  objectives 
and  included  in  Recommended  DoD  Actions 
above. 

hnplementing  these  concepts  will  increase 
productivity  of  all  DSAM  professionals  in 
centers  of  management  exceUence,  and  thus 
increase  productivity  substantially  in  the 
DSAM  process.  Ultimately,  it  can  increase 
productivity  even  more  throu^out  a 
seamlessly  integrated  IDSM  process  that 
covers  every  defense  system's  entire  life 
cycle. 

Endnotes 

^  The  terms  DSAM/IDSM  professional,  as 
used  in  this  paper,  include  any  government, 
contractor,  academic  or  other  knowledge 
worker  who  uses  DSAM  or  IDSM  knowl¬ 
edge  and  information  professionally  in  his  or 
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her  job  in  support  of  a  DSM  process. 

2  Information  management  technologies 
(IMTI  as  used  in  this  paper,  includes  most  of 
what  is  normally  defined  as  Information 
Technology  (IT),  and  also,  all  technologies 
that  use  IT  in  the  generatioi^  assembly, 
classification,  processing,  dissemination, 
selection  or  retrieval  of  knowledge  and 
information  for  use  in  management  (e.g., 
library  and  publications  technologies). 

^  President's  Blue  Ribbon  Commission  on 
Defense  Management.  1986.  In  Search  cf 
Excellence:  Final  Report  to  the  President 
(hereafter  referenced  as  Packard  Commis¬ 
sion  Report).  Washington,  DC.:  U.S.  Gov¬ 
ernment  Printing  Office,  June,  IIS  pages. 

*  Packard  Commission  Report,  pp.  52-71. 

*  Mosier,  Andrew  P.  1988.  "Past  Acquisi¬ 
tion  Improvements:  Not  Sufficient."  Pro¬ 
gram  Manager:  The  Journal  cf  the  Defense 
Systems  Management  College  (hereafter  ref¬ 
erenced  as  Program  Manager)  17  (May- 
June)  pp.  42-57. 

*  Mosier.  1985.  "A  Proposal  for  Research 
to  Improve  the  Productivity  of  Defense  Sys¬ 
tem  Acquisition  Managers  throughout  Gov¬ 
ernment  and  Industry."  Proceedings,  1985 
Federal  Acquisition  Research  Symposium, 
pp.  5-12.  See  referenced  Endnote  3  for  early 
research. 

^  Packard  Commission  Report,  pp.  xi,  xii. 

^  Mosier.  1989.  "Expediting  Acquisition 
Management  Knowledge  and  Information  to 
Centers  of  Management  Excellence.” 
Proceedings,  1989  Acquisition  Research 
Symposium,  pp.  301-306. 

^  Mathias,  John  R.,  and  Mosier,  Andrew  P. 
1991.  "Achieving  Excellence  in  Manage¬ 
ment  of  Defense  Systems."  Proceedings, 
1991  Federal  Acquisition  Research  Sympo¬ 
sium,  Vol.  n,  pp.  223-236.  See  pp.  227- 
228.  (Note:  The  last  two  pages  of  this  paper, 
inadvertently  omitted  fi’om  die  Proceedings, 
should  be  available  from  DSMC.  If  not.  Dr. 
Mosier  can  provide  them  as  pages  236a  and 
236b.) 

For  definitions  of  operational  effectiveness 
and  operational  suitability,  see  DoD  Instruc¬ 
tion  5000.2,  Defense  Acquisition  Manage¬ 
ment  Policies  and  Procedures  (February  23, 
1991),  Part  15,  Definitions,  p.  15-13. 


“  Mosier.  1988.  "Past  Acquisition  hn- 
provements:  Not  Sufficient."  pp.  47-50. 

Mosier.  1987.  "Getting  die  Jump  on  DoD 
Productivity."  Program  Manager  16  (Jufy- 
August)  pp.  18-26 

13  Ibid.  24-25. 

Mosier.  1989.  "New  Initiatives  and  Con¬ 
cepts  for  Increasing  Acquisition  Productiv¬ 
ity,"  Program  Manager  18  (March- April) 
pp.  24-34.  See  p.  24  for  productivity 
elements  firamework. 

^3  Ibid.  pp.  24-26  for  DoD  Initiatives.  Lack 
of  space  precluded  summary  of  new  concepts 
(pp.  26-28),  that  would  be  useful  in 
understandir^  DSAM  Knowledge  Systems 
concepts. 

Mosier.  1988.  "Past  Acquisition 
Improvements:  Not  Sufficient."  pp.  42-57. 

Mosier.  1989.  "Expediting  Acquisition 
Management  Knowledge  and  Information  to 
Centers  of  Management  Excellence."  pp. 
303-306. 

Mathm  and  Mosier.  1991.  "Achieving 
Excellence  in  Management  of  Defense 
Systems,"  pp.  225-231. 

Ibid.  pp.  230-233. 

30  Mosier.  1989.  "DoD  Actions  to  Increase 
Acquisition  Productivity."  Program  Manager 
18  (May  June)  pp.  60-72.  See  discussion  of 
DSAM  Taxonomy-Glossary,  pp.  62-66. 

3*  Ibid.  pp.  71. 

33  Ibid.  pp.  67-68,  70-71.  See  for  summary 
of  Other  DoD  Actions  recommended . 
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ABSTRACT 

There  are  multiple  approaches  to 
analyzing  risk  in  a  weapon  system  program. 
The  general  steps  of  a  cost  risk  analysis 
procedure  will  be  described; 

•  Preview  -  identify  key  risks  through 
execution  of  five  tasks: 

Define  the  purpose  of  the  risk 
analysis. 

Review  available  information 
concerning  the  program. 

Break  the  program  into  its 
components  at  some  specified 
level  of  detail  (i.e.  the  Cost 
Element  Structure-CES). 

Develop  a  list  of  risk  areas. 
Review  any  other  risk  analyses 
that  may  pertain  to  the  program. 

•  Perform  -  develop  the  risk  assessment 
through  completion  of  five  tasks. 

Develop  measurement  standards 
(percentage  of  cost  for  the  Low, 
Most  Likely,  and  High  applied  to 
the  Most  Likely  Cost  Estimate). 
Collect  available  data  for  each 
identified  risk  area  (i.e.  interview 
the  Technical  Risk  team  members 
as  well  as  the  Cost  Estimating 


team  members). 

Determine  the  range  of  possible 
outcomes  for  each  element  of  the 
CES  (i.e.  develop  a  probability 
distribution). 

Conduct  the  initial  modeling  and 
calculations. 

Identify  risk  items  needing 
additional  data  or  other  processes 
or  procedures  for  completion  of 
the  assessment. 

•  Present  -  Display  the  results  in  a 
manner  that  allows  reviewers  to 
understand  the  analysis. 

The  presentation  will  describe  currently 
used  alternative  methodologies  including 
simulation,  method  of  moments,  and 
analytical  formulas  to  complete  the  risk 
analysis  process.  These  are  too  lengthy  to 
include  in  this  paper. 
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INTRODUCTION 


Risk  assessment  is  a  comprehensive 
and  structured  process  where  the  risk  of  a 
particular  course  of  action  is  estimated.  We 
must  understand  the  program  well  enough  to 
identify  unplanned  and  unknown  events 
potentially  facing  a  program,  their  nature, 
their  probability  of  occurrence,  and  the 
consequences  of  their  occurrence.  In 
mathematical  terms,  risk  analysis  models 
determine  and  use  probabilities  and 
probability  distributions  to  assess  the 
combined  impact  of  risk  on  the  program. 
Every  model  manipulates  the  data 
differently.  Selection  of  risk  areas  and 
collection  of  data  for  input  into  a  selected 
model  and  the  subsequent  presentation  of  the 
model’s  outputs  are  the  core  activities 
involved  in  risk  analysis. 

Basic  Modeling  Approaches.  Risk 
analysis  is  an  important  factor  in  the 
decision  making  process  for  complex 
systems.  The  process  of  risk  analysis 
provides  additional  insight  for  decision 
makers  through  the  collection  and 
interpretation  of  quantitative  data.  It  is 
important  to  remember  that  decisions  facing 
managers  of  systems  will  be  made  under 
conditions  of  less  than  perfect  knowledge 
about  the  future.  No  analyst  nor  program 
staff  can  ever  assemble  the  perfect 
knowledge  base  about  the  program.  A 
process  to  make  quantitative  models  to  link 
expert  judgements,  historical  facts,  and 
program  plans  to  possible  outcomes  needs  to 
be  established.  Figure  1  shows  there  are 
three  basic  stages  in  approaching  the  risk 
analysis: 

•  preview, 

•  perform,  and 

•  present. 


THE  THREE  COST  RISK  ANALYSIS 
STAGES 

PREVIEW 

•  Define  the  purpose  of  the  risk 
analysis 

•  Review  available  information 
concerning  the  program 

•  Determine  components  to  be 
analyzed 

•  Develop  list  of  risk  areas 

•  Review  other  pertinent  risk 
analyses 

PERFORM 

•  Develop  measurement  standards 

•  Collect  data  for  risk  areas 

•  Develop  distribution  information 
for  each  component  to  be 
analyzed 

•  Model  the  data 

•  Follow  up 

PRESENT 

•  Document  the  results 

•  Perform  sensitivity  analysis 


Figure  1 .  The  Three  Cost  Risk  Analysis 
Stages  Follow  a  Logical  Work  Flow' 

Selection  of  Model.  There  is  no  one 
right  model  to  conduct  cost  risk  analysis. 
Because  each  program  office  estimate  uses 
several  estimating  techniques  to  compute  the 
estimate,  a  risk  analysis  may  have  to  rely  on 
more  than  one  risk  analysis  model  or 
technique  to  be  effective.  The  selection  and 
use  of  a  particular  risk  analysis  model 
depends  on  the  amount  and  type  of 
information  available  for  the  analysis  as  well 
as  model  availability,  the  skills  and 
availability  of  trained  analysts,  and  the  level 
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The  risk  analyst  needs  to  become  a 
source  for  providing  information  on  a  wide 
range  of  possible  outcomes  of  the  program. 
The  analyst’s  basic  task  is  to  establish  a 
rational  model  that  connects  probabilities 
developed  from  expert  sources  with  program 
plans  and  alternatives.  As  a  result  of  the 
risk  analysis  process,  the  analyst  will 
develop  a  report  that  provides  information. 
The  analyst  should  be  aware  that  the  report 
will  be  used  as  a  basis  for  communication 
between  many  parts  of  the  program  and  with 
decision  makers.  As  part  of  the 
communication  process,  many  individuals 
may  provide  comments  and  other  forms  of 
constructive  criticism.  This  form  of 
feedback  provides  the  analyst  with  a 
framework  for  revising  the  groundrules  and 
assumptions  comprising  the  risk  analysis 
model.  Some  of  the  comments  provided  as 
feedback  may  not  be  included  in  a  revision 
to  the  model.  However,  the  analyst  needs 
to  provide  this  feedback  to  the  decision 
maker,  possibly  in  another  form  like 
sensitivity  analysis  or  as  a  part  of  the 
discussion  in  the  final  report, 
of  information  needed  by  the  decision 
makers.  The  risk  analysis  model(s) 
selected  should  be  able  to  provide  Air 
Force  and  Government  management  with 
information  that  can  be  used  in  making 
better  informed  decisions  concerning  what 
may  happen  and  what  action  should  be 
taken. 

General  Risk  Models.  The  general 
risk  models  develop  cost  and  risk 
information  about  a  program  based  on 
historical  data  of  previous  or  similar 
programs.  They  use  statistical  relationships 
between  cost  and  other  parameters 
describing  the  system.  Modeling  techniques 
in  this  group  include  considering  risk 
through  the  use  of  Cost  Estimating 
Relationships  (CERs).  Several  hardware 
firms  have  their  own  internal  models.  One 


related  general  risk  model  is  the  Range 
Estimating  Program. 

Stochastic/Probabilistic  Models. 
Stochastic/probabilistic  models  develop 
probability  distributions  for  cost  elements  or 
whole  programs  based  on  statistical, 
mathematical,  and  simulation  techniques. 
Examples  of  this  type  of  model  include  the 
Air  Force  Risk  Model,  the  MCR  Risk 
Model,  the  Army  VERT  (Venture 
Evaluation  and  Review  Technique)  model, 
PERT  (Program  Evaluation  and  Review 
Technique)  analysis,  CPM  (Critical  Path 
Method)  analysis,  and  other  sophisticated 
variants  of  these  approaches.  These  models 
allow  for  the  aggregation  of  cost  and  risk 
values  for  the  entire  program. 

RISK  ANALYSIS  PROCEDURE 

The  cost  analysis  team  must  have  a 
well  defined  plan  to  conduct  a  risk  analysis. 
The  risk  analysis  is  a  necessary  component 
of  the  complete  cost  analysis.  Regardless  of 
the  results  of  the  risk  analysis,  the  process 
and  product  serves  to  provide  additional 
information  to  decision  makers.  Another 
benefit  of  the  risk  analysis  is  that  it  serves  to 
allow  informed  people  from  all  parts  of  the 
program  to  participate  in  making  an  estimate 
of  the  future  range  of  outcomes  facing  the 
program.  The  input  of  experts 
knowledgeable  about  the  program  can  be 
used  to  produce  valuable  information  about 
future  possible  outcomes  in  the  program. 
There  are  three  basic  stages  in  conducting 
the  cost  risk  analysis.  The  following 
paragraphs  discuss  each  of  these  stages. 

PREVIEW  STAGE.  The  preview 
stage  consists  of  identifying  key  risks 
through  five  tasks.  These  tasks  are: 

•  Define  the  purpose  of  the  risk  analysis. 
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•  Review  available  information 

concerning  the  program, 

•  Break  the  program  into  its  components 

at  some  specified  level  of  detail, 

•  Develop  a  list  of  risk  areas,  and 

•  Review  any  other  risk  analyses  that 

may  pertain  to  the  program. 

Define  the  purpose  of  the  risk 
analysis.  In  preparing  the  cost  analysis 
plan,  the  team  chief  needs  to  clearly  specify 
the  objectives  of  the  risk  analysis.  The  risk 
analyst  needs  to  understand  the  potential 
problems  facing  the  analysis,  the  needs  of 
the  decision  makers  for  information,  goals 
of  the  cost  analysis  team,  and  the  objectives 
of  the  risk  analysis.  The  analyst  serves  as  a 
conduit  of  information  from  expert  sources 
to  the  decision  maker. 

(1)  One  fundamental  consideration  in 
defining  the  purpose  of  the  risk  analysis  is 
that  no  future  activity  should  be  included  in 
the  risk  analysis  unless  the  analysis  results 
could  lead  to  a  decision  to  alter  the  present 
course  of  action.  Risk  analysis  in  the  cost 
analysis  process  should  not  be  considered 
only  as  a  mandatory  requirement.  The 
purpose  of  conducting  the  risk  analysis  must 
include  the  possibility  that  the  results  of  the 
process  may  affect  the  present  direction  of 
the  program  and  prepare  the  program  to  face 
an  uncertain  future. 

(2)  Another  consideration  in  defining 
the  purpose  of  the  risk  analysis  is  the  time 
focus  of  the  analysis.  There  may  be  specific 
areas  of  the  program,  having  a  specific 
timeframe.  If  the  decision  makers  have 
communicated  concern  for  a  specific  set  of 
activities  leading  to  a  program  milestone, 
the  program  timeframe  surrounding  those 
activities  needs  to  be  investigated. 
Additionally,  the  farther  out  in  the  future  of 
the  program  life  cycle  an  area  of  risk  may 
reside,  the  more  uncertain  it  will  be.  The 
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analyst  needs  to  consider  the  overall  time 
schedule  of  events  when  assessing  the  levels 
of  uncertainty.  Generaly,  an  event  will  have 
more  uncertainty  the  further  in  the  future  it 
occurs.  Events  closer  to  the  present  should 
have  less  uncertainty  associated  with  them. 

(3)  The  fundamental  task  of  the  risk 
analysis  is  to  evaluate  the  program  in  terms 
of  achieving  its  plan  with  the  resources 
allocated.  The  cost  analysis  plan  will 
include  information  that  will  form  the  smdy 
requirements  and  objectives.  Understanding 
the  detailed  requirements  and  objectives  is 
necessary.  During  the  study  these  may 
change  because  of  new  information  or 
direction,  so  their  understanding  is 
necessary. 

(4)  When  establishing  the  risk 
analysis  objectives,  it  is  necessary  to  define 
the  boundary  between  the  internal  program 
and  the  external  factors  affecting  the 
program.  When  deciding  what  needs  to  be 
included  in  the  risk  analysis,  the  choices 
may  be  made  subjectively.  The  choices  will 
be  influenced  by  such  factors  as  lack  of 
pertinent  data,  schedule  or  personnel 
constraints,  and  uncertainties  associated  with 
selecting  an  appropriate  analysis  model. 
Therefore,  the  thorough  statement  and 
understanding  of  the  risk  analysis  objective 
is  critical.  Once  the  objective  is  understood, 
the  analysis  boundaries  can  be  determined 
and  the  specific  areas  of  risk  can  be 
evaluated.  An  example  of  an  external  factor 
that  should  be  handled  separately  is  funding 
limits,  which  could  constrain  test  assets  and 
result  in  schedule  slippage. 

(5)  When  stating  the  risk  analysis 
objective,  use  past  experience  and 
knowledge  of  the  overall  defense 
environment  and  operation.  Avoid  stating 
the  objective  in  too  broad  terms.  Broad 
objective  statements  require  many 


assumptions.  The  use  of  too  many 
assumptions  may  weaken  the  focus  on 
known  information  or  add  unnecessary  bias. 

(6)  When  the  number  of  assumptions 
increases  in  an  analysis,  a  higher  level  of 
uncertainty  can  be  introduced  on  the 
conclusions  resulting  from  the  analysis.  If 
the  risk  analysis  objective  is  stated  in  limited 
terms,  it  may  prevent  important  facets  of  the 
program  from  being  investigated. 

(7)  Clearly  list  all  assumptions.  The 
assumptions  need  to  be  justified  by  historical 
or  other  factual  evidence.  If  there  is  no 
factual  basis  for  an  assumption,  the  rationale 
for  requiring  the  assumption  should  be 
stated.  There  may  be  valid  reasons  for 
including  such  an  assumption,  including 
convenience  in  modeling,  generally  accepted 
positions  affecting  the  program,  or  lack  of 
sufficient  factual  basis.  The  statement  of  the 
background  of  these  assumptions  will 
highlight  areas  where  analysis  errors  or  bias 
may  be  introduced. 

(8)  Constraints  on  the  objective  are 
necessary  for  a  variety  of  reasons.  Without 
constraints  the  analysis  would  be  open- 
ended,  preventing  a  conscientious  evaluation 
of  the  program’s  risk  areas.  Without 
boundaries  the  analyst  may  also  be  required 
to  evaluate  the  various  risk  areas  without 
considering  realistic  future  bounds  on  the 
program.  Constraints  involving  budget, 
schedule  and  technology  need  to  be 
developed  and  used. 

Review  available  information 
concerning  the  program  including  the 
current  cost  estimate.  Determine  the 
underlying  assumptions,  the  Work 
Breakdown  Structure,  items  of  uncertainty 
about  the  cost  elements,  the  proposed 
development,  production,  and  operating 
phase  assumptions,  and  schedule.  The 


analyst  should  ask:  "what  kinds  of  activities 
may  have  a  range  of  possible  outcomes? " 

(1)  The  analyst  needs  to  understand 
the  components,  their  functions,  and  the 
interrelationships  of  the  program  under 
evaluation.  The  system  needs  to  be 
described  in  terms  that  allow  the  risk 
analysis  to  be  performed  in  accordance  with 
the  risk  analysis  objectives. 

(2)  The  boundaries  of  the  analysis 
should  include  minimum  and  maximum 
feasible  conditions.  The  analyst  should 
exercise  care  when  evaluating  each  risk  area 
and  ensure  that  the  analytical  approach  does 
not  exceed  the  boundaries.  If  the  result  of 
the  analysis  appears  to  exceed  the 
boundaries,  the  analyst  needs  to  inform  the 
other  members  of  the  team  so  an  evaluation 
can  be  made. 

(3)  The  determination  of  the 
boundaries  is  not  solely  the  responsibility  of 
the  risk  analyst.  A  multi-disciplinary  team 
should  work  together  to  develop  the 
boundaries. 

(4)  The  analyst  should  be  prepared  to 
modify  the  items  being  evaluated  during  the 
conduct  of  the  risk  analysis. 

(5)  The  selection  of  the  items, 
boundaries,  and  underlying  assumptions 
should  be  combined  into  a  workable  model. 
The  combination  of  items  should  represent 
a  feasible  set  of  possible  outcomes  that 
reflect  the  internal  and  external  program 
factors  discussed  below. 

Break  the  program  into  its 
components  at  some  specified  level  of 
detail.  The  risk  analysis  may  not  be 
conducted  at  the  same  level  the  program  was 
estimated.  Depending  on  how  the  estimate 
was  prepared,  there  may  not  be  a  need  to 
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analyze  to  the  lowest  level  of  the  program 
structure.  There  may  be  some  aggregation 
necessary  for  the  risk  analysis.  The  cost 
analysis  plan  should  provide  information 
concerning  the  level  of  detail. 

Develop  a  list  of  risk  areas. 
Determine  an  initial  list  of  program  elements 
that  are  potentially  risky.  There  may  be 
some  program  elements  that  have  little  or  no 
risk  associated  with  the  value  expressed  in 
the  estimate.  These  elements  still  are  part 
of  the  total  program  cost,  but  their 
uncertainty  may  be  slight  or  not  existent. 
Additionally,  the  ICA  Plan  will  contain 
some  areas  that  are  directed  for  risk 
analysis.  Make  a  gross  assessment  of  the 
trend  in  program  cost  and  schedule  for  each 
risk  area  as  it  may  develop.  Estimate  if  and 
how  each  individual  risk  area  may  depend 
on  the  outcome  of  one  or  more  other  areas 
of  the  project  and  the  extent  of  the 
dependency  (complete,  partial).  The 
analyst  should  ask;  "what  is  involved  in  the 
project  that  may  not  occur  as  estimated  or 
planned  in  the  baseline?"  In  determining  the 
risk  areas  the  analyst  needs  to  consider 
external  and  internal  factors  that  will  form 
the  environment  for  the  program.  Refer  to 
Figure  2  for  a  summary. 

(1)  Analyzing  external  factors 
affecting  the  program  normally  requires 
information  collected  from  expert  sources 
knowledgeable  about  both  the  program 
structure  and  the  external  environment. 

(2)  In  considering  internal  factors,  the 
analyst  needs  to  consider  the  differences  in 
the  program  that  make  its  future  outcome 
differ  from  past  programs.  New 
technology,  new  designs,  or  different 
manufacturing  processes  will  involve 
uncertainty  and  make  the  program  different 
from  previous  similar  programs.  The 
analyst  needs  to  estimate  the  impact  of  the 


changes  and  anticipated  improvements. 
These  estimates  need  to  be  translated  into 
quantitative  data  for  inclusion  in  the  risk 
analysis  model. 

Review  any  other  similar  risk 
analyses.  Review  other  risk  analyses  that 
may  have  been  conducted  for  this  or  similar 
programs  as  applicable.  In  this  step, 
determine  if  the  methodology  used  for 
previous  analyses  needs  to  be  modified  for 
use  in  this  analysis.  The  purpose  of 
reviewing  other  risk  analyses  is  to  see  if 
anything  can  be  learned  from  the  histories  of 
other  similar  programs.  The  purpose  of  the 
risk  analysis  is  to  provide  insight  into  the 
future.  The  analyst  needs  to  insure  that  any 
trends  or  tendencies  seen  in  other  programs 
are  free  of  unique  occurrences  or  historical 
operations.  The  exclusion  of  the  rare 
previous  events  from  the  trends  will  clear  up 
the  analysis  of  potential  bias  from  events 
that  probably  will  not  happen  in  the  future 
of  this  program.  The  analysis  of  previous 
histories  requires  knowledge  and  insight  of 
the  previous  programs  and  an  understanding 
of  the  current  program. 

PERFORM  STAGE.  The  steps  for 
performing  the  next  phase  of  the  risk 
assessment  are  as  follows; 

•  Develop  measurement  standards. 

•  Collect  available  data  for  each 
identified  risk  area, 

•  Determine  the  range  of  possible 
outcomes  for  each  item  (develop  a 
distribution), 

•  Conduct  the  initial  modeling  and 
calculations,  and 

•  Identify  risk  items  needing  additional 
data  or  other  processes  or  procedures 
required  for  completion  of  the 
assessment. 
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Figure  2.  The  Risk  Analysis  Needs  to 
Consider  External  and  Internal  Program 
Factors 


Develop  measurement  standards.  In 
order  to  provide  a  basis  for  comparing  and 
analyzing  the  risk  assessment  results,  a  set 
of  measurement  standards  needs  to  be 
prepared  in  advance  of  the  analysis. 
Measurement  standards  are  also  known  as 
Measures  of  Effectiveness  (MOE).  These 
should  be  in  terms  of  dollars,  hours, 
pounds,  feet/seconds,  miles/hours,  degrees, 
etc.  All  possible  conditions  in  the  program, 
their  potential  interactions,  and  their 
relevance  to  the  study  objective  must  be 
evaluated  thoroughly. 

(1)  If  the  analyst  is  faced  with  many 
areas  of  risk  in  the  analysis,  a  sensitivity 
analysis  should  be  performed.  The 


sensitivity  analysis  can  provide  insight  into 
the  relative  weighting  of  each  risk  area  on 
the  overall  program.  Not  all  of  the  potential 
risk  areas  will  be  needed  in  the  evaluation  of 
the  program’s  overall  risk  levels.  Some 
areas  of  the  program  interact  more 
significantly  with  other  areas  than  do  the 
remaining  areas.  These  less  significant 
areas  may  not  be  as  sensitive  to  the  overall 
level  of  risk  to  the  program.  However, 
without  first  identifying  and  ordering  all 
potential  risk  areas,  it  would  be  impossible 
to  distinguish  precisely  which  potential  risk 
areas  needed  to  be  included. 

(2)  Some  of  the  risk  areas  can  be 
controlled  early  in  the  life  of  the  program. 
Some  risk  areas  cannot  be  controlled  until 
later  in  the  life  of  the  program.  The 
criticality  of  risk  areas  can  change  during 
the  program’s  life.  The  analyst  should  be 
aware  of  these  potential  changes. 

Collect  data  for  each  identified  risk 
area.  This  step  is  subject  to  the  amount  of 
time  and  data  collection  resources  available. 
Additionally,  the  available  data  may  not  be 
in  a  form  you  can  use,  or  it  may  not  be 
applicable  to  the  risk  analysis  activity  at 
hand.  It  takes  judgment  and  knowledge  of 
the  program  to  make  informed  decisions 
about  the  data  and  the  effort  required  to 
collect  and  use  it.  Current  or  available 
models  may  be  modified  to  permit  use  of  the 
results  of  the  risk  assessment  by  determining 
what  important  assumptions  and  limitations 
must  be  imposed  to  provide  a  reasonable 
analysis.  Determine  the  consequences  of 
each  possible  risk  area  actually  occurring  in 
the  project  and  formulate  a  feasible  response 
that  would  lead  to  mitigating  the  risk  area. 
Determine  the  response  in  terms  of  the 
common  denominator:  "is  this  a  problem 
that  time  or  money  could  help?" 

(1)  The  analyst  needs  to  collect 
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(1)  The  analyst  needs  to  collect 
information  on  the  entire  setting  of  the  risk 
area. 


Figure  3.  Consider  All  Interfaces  When 
Collecting  Risk  Related  Data 

(2)  The  individual  risk  item  is 
defined  by  the  requirements,  constraints, 
and  interface  characteristics  under  which  the 
system  will  ultimately  operate.  This 
individual  risk  item  affects  and  is  affected 
by  other  individual  risk  items  in  the  system. 
The  individual  risk  items  have  in  common 
certain  related  development  or  operational 
functions.  The  individual  risk  item  also 
affects  and  is  affected  by  the  structure 
intended  to  support  the  item.  In  addition, 
the  individual  risk  item  is  affected  by 
external  factors  (e.g.,  availability  of  test 
facilities,  etc.).  The  analyst  needs  to  collect 
information  on  all  these  interfaces. 


Determine  the  range  of  possible 
outcomes  for  each  item.  In  this  step  the 
analyst  will  develop  sufficient  information  to 
enable  a  description  of  the  probability 
distributions  for  the  selected  elements.  The 
analyst  will  need  to  consult  with  other 
members  of  the  ICA  team  and  possibly  other 
experts  for  information  on  the  range  of 
possible  costs  for  the  elements,  what  would 
cause  the  costs  to  vary,  and  the  likelihood 
that  particular  cost  levels  will  occur.  This 
step  involves  knowledge  of  the  inter¬ 
workings  of  the  program  as  well  as  of  the 
individual  items  under  study.  Select  a 
model  or  analysis  technique  that  appears 
well-suited  for  the  analysis  task.  Based  on 
the  information  collected,  you  may  select 
different  approaches  for  different  elements. 

Conduct  initial  model  computations 
and  interpretation.  Use  the  selected  risk 
model  or  technique  and  the  current  inputs. 
Compute  the  probability  distribution. 
Analyze  the  results  for  feasibility  and 
rationality.  Update  the  model  and  input  data 
and  reiterate  the  process  as  appropriate  until 
the  results  are  satisfactory. 

(1)  A  risk  analysis  model  is  a  logical, 
mathematical  representation  of  the  specific 
program  areas  and  their  interdependencies. 
It  takes  the  form  of  mathematical  equations. 
The  model,  when  properly  constructed, 
quantifies  the  range  of  possible  outcomes 
and  their  associated  probabilities  for  each 
item.  The  model  should  also  provide  the 
quantification  of  risk  due  to  the  individual 
items  interacting  to  form  the  entire  program. 
There  is  a  great  amount  of  interaction 
between  the  individual  items  under  analysis. 
Constructing  the  risk  model  to  portray  this 
interaction  requires  the  combined  efforts  of 
experts  and  other  sources. 

(2)  All  assumptions  used  in 
constructing  the  model  need  to  be  stated 
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clearly.  They  should  be  supported  by 

factual  evidence  if  possible.  If  no  factual 
evidence  is  available,  the  underlying 

rationale  for  the  use  of  the  assumption 
should  be  known. 

(3)  The  risk  analysis  model  must 

cover  the  program  adequately.  Not  every 
potential  risk  item  needs  to  be  included  to 
represent  the  full  spectrum  of  program  risk. 
No  model  can  perfectly  nor  completely 

duplicate  the  system.  It  is  imperative  that 
the  analyst  attempt  to  cover  the  entire 
spectrum  of  the  program. 

(4)  The  analyst  cannot  avoid 

modeling  uncertainties  facing  the  program. 
Technical  or  schedule  risk  may  be  difficult 
to  collect  and  analyze,  but  they  must  be 
considered  in  the  model.  Modeling  the 
program  in  this  way  can  be  ineffective.  The 
probability  of  guessing  correctly  for  every 
uncertain  area  is  very  small.  The  use  of 
selected  modeling  approaches  can  reduce  the 
effects  of  uncertain  or  ambiguous  data. 

(5)  The  construction  of  the  model  is 
directly  affected  by  the  type  and  amount  of 
data  collected.  Data  are  needed  to  support 
the  groundrules  and  assumptions,  define  the 
distributions,  and  the  interfaces.  Missing 
data  items  may  prevent  complete  model 
utility.  The  analyst  should  consider  the 
benefit  of  collecting  additional  data  versus 
the  cost. 

(6)  Validation  of  the  model  requires 
consideration  of  several  aspects  including 
the  following  items: 

•  Sensitivity:  are  the  changes 

consistent  with 
expectations? 

•  Suitability:  does  the  model 

answer  the  right 
questions? 


•  Consistency:  are  the  results 

consistent  across  the 
entire  range  of  the 
distributions? 

•  Criticality:  do  minor  changes  in 

the  groundrules  and 
assumptions  cause 
major  changes  in  the 
results? 

•  Applicability:  does  the  model 

require  data  or 
computational 
resources  not 
currently  available? 

The  analyst  should  consider  the 

answers  to  questions  such  as  these  to 
determine  areas  of  the  model  that  may  need 
modification. 

Identify  risk  items  needing  additional 
data.  This  step  comes  from  the  careful 
analysis  and  use  of  the  collected  program 
data.  While  performing  the  risk  analysis, 
the  analyst  may  find  it  necessary  to  perform 
additional  analyses  or  use  a  different 
approach  for  one  or  more  items.  The 
analyst  should  keep  in  mind  all  the  available 
resources,  models,  tools,  and  other  means  of 
support  so  they  can  be  used  to  conduct 
additional  analyses  as  necessary.  The 
analyst  should  ask  questions  like:  "what 
could  cause  the  estimated  cost  or  schedule  to 
change?"  or  "what  could  happen  if  these 
things  occur?"  to  investigate  possible  cause 
and  effect  relationships  for  applicability  to 
this  situation. 

PRESENT  STAGE.  The  risk  analysis 
is  a  management  decision  tool.  Usually,  the 
direct  output  of  the  conduct  of  a  risk 
analysis  is  not  in  a  form  easily  used  or 
understood  by  management.  The  results  of 
the  model  or  technique  could  be  a  series  of 
descriptive  statistics  or  rows  and  columns  of 
numbers.  The  decision  makers  may  not  be 
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sensitive  to  the  message  of  the  data  in  those 
forms.  The  analyst  needs  to  reformat  the 
data  into  a  form  useful  to  the  decision  maker 
so  that  the  presentation  of  the  results  will 
increase  the  utility  of  the  tool.  To  facilitate 
this  presentation,  the  cost  analysis  process 
has  specific  documentation  formats  that  are 
to  be  followed  which  convey  necessary 
information  in  a  manner  useful  to  Air  Force 
and  other  Government  managers.  After 
conducting  the  final  computation,  the 
documentation  is  finalized  to  accurately 
reflect  the  inputs,  outputs,  and  analyses  of 
the  results. 

CONCLUSIONS 

A  structured  process  for  conducting  a 
cost  risk  analysis  has  been  presented.  All 
that  remains  is  to  use  one  of  the  available 
models  to  aggregate  and  present  the  risk 
distributions. 
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ABSTRACT 

Electronic  sources  of  procurement 
information  are  appearing  everywhere: 
bulletin  boards,  CD-ROMs,  on-line 
services,  etc.  However,  there  is  still  not 
widespread  use  of  these  services  by  the 
professionals  who  will  benefit  from  these 
capabilities,  the  field  contracting  officers. 

The  paper  looks  at  the  impediments  which 
exist  to  implementation  of  the 
Procurement  Knowledge  Network  (PKN): 
an  easy-to-use  information  system  which 
will  allow  procurement  information  to 
flow  freely  from  its  source  to  eveiy 
procurement  professional  who  needs  it. 

The  impediment  is  not  the  technology: 
information  networks  exist  as  illustrated  by 
the  government’s  increasing  use  of 
electronic  mail  (E-Mail)  to  communicate 
among  its  professional  staff.  Value  added 
networks  (VANs)  have  also  created  broad- 
based  capabilities  including  bulletin 
boards  and  electronic  data  interchange 
(EDI).  CD-ROM  technology  has  advanced 
to  the  point  that  an  information  source 
can  quickly  and  inexpensively  master  disks 
containing  large  amounts  of  data  for 
distribution.  Government  organizations 
such  as  the  Government  Printing  Office 
(GPO)  have  created  electronic  publishing 
for  the  Government  facilities  to  use  to 
meet  their  publishing  needs. 

The  impediment  is  jiiiJ  the  information 
source:  virtually  all  Government  agencies 
which  create  procurement  information  use 


automation  to  create  that  information. 
The  various  agencies  make  use  of 
wordprocessing,  desk  top  publishing, 
database  management  systems,  and  a 
variety  of  automated  tools  to  manage  vast 
amounts  of  information  within  their  own 
office  environments. 

The  impediment  is  noj  the  potential  user: 
most  procurement  offices  are  using,  at  a 
minimum,  personal  computers  to  perform 
such  activities  as  order  generation, 
solicitation  and  contract  preparation, 
procurement  tracking,  and  regulatory 
research.  Many  are  also  disseminating 
their  information  through  bulletin  boards 
and  increasingly  through  EDI. 

Problems,  however,  do  still  exist  in  the 
infrastructure  to  support  government-wide 
procurement  information  exchange.  Each 
procurement  information  source  is  specific 
to  the  organization  which  generated  it  and 
exchanging  information  between  any  two 
organizations  is  cumbersome.  There  is 
little  appreciation  among  procurement 
professionals  of  the  many  government 
standards  which  support  the  movement  of 
data  between  systems. 

These  standards  do  not  necessary  mean 
that  all  organizations  which  generate 
information  must  operate  using  the  same 
computer  platform  or  follow  the  same 
processes.  Standards  such  as  the 
Standardized  Generalized  Markup 
Language  (SGML)  impose  a  discipline  for 
the  transfer  of  information  from  one 
organization  to  another:  interoperability. 
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Standards  can  also  assist  the 
implementation  process  by  minimizing  the 
number  of  access  methods  and  search 
tools  that  individual  users  must  use. 
Currently,  procurement  professionals  in 
Government  field  offices  must  know 
where  information  exists;  what  telephone 
number  to  use  to  access  the  information; 
what  communication  protocol  to  use  to 
communicate  with  the  information  source; 
and  finally  how  to  navigate  around  the 
information  source  to  find  the  necessary 
data.  Many  in  the  end,  just  don't  bother: 
instead  relying  on  the  printed  books  in  their 
offices  or  telephone  calls  to  other 
contracting  professionals. 

This  paper  examines  the  issues  which  must 
be  addressed,  today,  if  a  Procurement 
Knowledge  Network  (PKN)  is  to  be 
effective  by  the  end  of  the  1990s. 

The  electronic  revolution  has  begun,  as 
witnessed  by  the  number  of  electronic 
procurement  information  resources  which 
have  arisen  over  the  past  two  years. 

Considerable  work  still  needs  to  be  done 
to  streamline,  standardize,  and  reach  the 
goals  of  equal  and  easy  electronic  access 
to  all  procurement  information  by  every 
Government  procurement  official. 

INTRODUCTION 

The  Government  procurement  process  is 
time-consuming  and  complex.  Individual 
contracting  officers  must  comply  with 
applicable  Federal  statutes,  the  Federal 
Acquisition  Regulation  (FAR),  and 
department-wide,  agency-wide,  and  often 
local  implementations  of  these 
requirements.  Individual  agencies  must 
also  interact  closely  with  other  government 
agencies,  such  as  the  Small  Business 
Administration  (SBA)  and  the 


Department  of  Labor  (DOL),  to  obtain 
the  information  necessary  for  any 
individual  procurement. 

Obtaining  information  is  one  of  the  major 
contributors  to  the  slowness  of  the 
procurement  process,  and  as  such,  the 
availability  of  electronic  sources  of 
information,  readily  available  when 
needed,  could  significantly  help  streamline 
the  government  procurement  process. 

Electronic  sources  of  procurement 
information  (regulations,  schedule 
information,  etc.)  are  becoming  readily 
available.  Some  are  being  distributed  by 
the  government,  such  as  by  the  General 
Printing  Office,  others  are  available 
commercially  from  private  sector 
publishers.  To  be  effective,  these 
information  sources  must  be  readily 
available  to  the  contracting  officer  using  a 
concept  such  as  the  Procurement 
Knwoledge  Network  (PKN).  (Reference 
1). 
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AUTOMATED  PROCUREMENT 
SYSTEMS 

Many  procurement  offices  have  automated 
their  day-to-day  activities.  The  typical 
procurement  office  has  developed  tools  to 
support  the  paper-based  Federal 
procurement  process.  These  tools  vary  in 
complexity  and  also  vary  in  the  extent  to 
which  they  improve  the  efficiency  of  the 
process. 

Some  agencies  are  merely  using 
wordprocessing  software  on  personal 
computers  to  generate  Purchase  Orders 
and  Delivery  Orders.  Other  agencies  have 
implemented  large-scale  automated 
procurement  management  systems  to  meet 
their  requirements  for  procurement 
tracking,  small  purchase  processing, 
solicitation  and  contract  preparation, 
vendor  management,  regulatory  research, 
and  Federal  reporting  requirements. 

These  systems  are  primarily  designed  to 
automate  the  processes  within  the 
Procurement  Office,  with  paper  being  the 
main  input  to  the  system,  and  paper  being 
the  main  output  to  the  system. 


There  are  no  standards  for  autr  nated 
procurement  systems  within  the  eueral 
government,  neither  i  .here  a  certification 
process  as  there  is  for  government 
F'  ancial  systems. 

As  such,  there  are  many  different 
implementations  of  automated 
procurement  systems.  These  include  the 
functions  that  are  automated,  the  purpose 
of  the  system,  the  computer  platform  used, 
etc.  Most  contracting  officers  receive 
some  training  in  internal  automated 
systems  and  are  becoming  increasingly 
comfortable  working  in  the  specific 
environment  of  their  office.  Many 
frequently  use  E-Mail  to  communicate 
with  others  in  their  agency. 

The  major  change  in  these  automated 
procurement  systems  is  the  move  towards 
integrated  systems:  systems  which 

combine  many  functional  capabilities  into 
a  seamless  environment  where  information 
flows  as  the  procurement  progresses. 

The  biggest  advantage  of  these  systems  is 
that  data  flows  from  one  function  to 
another,  without  the  need  for  rekeying. 
For  example,  information  such  as  a 
contract  type  or  dollar  value  entered  into 
the  system  to  prepare  an  award  document, 
can  be  automatically  transferred  to  the 
Federal  Procurement  Data  System  (FPDS) 
for  later  FPDS  reporting. 

AGENCY-WIDE  INTEGRATED 
FINANCIAL  SYSTEMS 

OMB  Circular  A-127,  Financial 
Management  Systems,  has  placed 
requirements  on  agencies  to  implement 
agency-wide  integrated  financial  systems. 
(Reference  2).  After  payroll,  the 
Procurement  Office  is  usually  the  largest 
generator  of  financial  transactions.  As 
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such,  the  procurement  system  is  seen  as 
part  of  an  agency’s  integrated  financial 
system. 

Integration  can  occur  at  two  levels.  The 
first  level  is  when  integration  occurs  by 
the  electronic  transfer  of  financial 
information  from  the  procurement  system 
to  the  financial  system:  this  is  usually 
referred  to  as  interoperability. 

The  second  level  of  integration  occurs 
when  the  agency  uses  the  same  database 
or  system  for  both  the  financial  and 
procurement  functions.  This  full 
integration  becomes  much  more 
complicated  since  the  operating 
organizations  have  to  become  integrated 
in  their  method  of  working.  Also,  fully 
integrated  systems  require  more 
processing  power  and  management 
attention  than  that  needed  by  two 
separate  systems 

Most  agencies  are,  therefore, 
implementing  solutions  which  meet  the 
needs  of  the  OMB  Circular  A-127,  yet 
maintain  the  separateness  and 
functionality  of  the  procurement  and 
financial  systems.  The  individual  systems 
communicate  with  each  other  through  a 
series  of  data  transfers,  agreed  upon  by 
procurement  and  finance. 

ELECTRONIC  REQUISITIONING 

Another  area  where  considerable  progress 
has  been  made  is  in  the  area  of  electronic 
requisitioning:  the  use  of  automation  to 
assist  in  the  preparation  of  user 
requirements  and  the  transmission  of 
those  requirements  to  the  procurement 
office. 

The  primary  value  of  electronic 
requisitioning  is  that  it  can  reduce  the 


lead  time  in  the  overall  acquisition 
process. 

The  automated  system  can  help  the 
requisitioner  prepare  the  procurement 
package,  ensuring  that  is  complete  and 
accurate  before  it  reaches  the 
procurement  office. 

Many  electronic  requisitioning  systems 
also  allow  requisitions  to  be  routed 
electronically  through  such  offices  as 
program  management,  budget,  information 
resource  management  (IRM),  property, 
etc.  Additional  steps  may  also  be 
implemented  for  special  conditions  (e.g., 
the  use  of  hazardous  materials). 

In  some  agencies,  electronic  requisitioning 
is  an  extension  of  the  procurement  system. 
One  example  is  the  Department  of 
Veterans  Affairs  (VA).  An  automated 
system  termed  IFCAP  (Integrated  Funds 
Distribution,  Control  Point  Activity, 
Accounting,  and  Procurement)  is  used 
throughout  the  VA  medical  centers  and 
other  facilities.  Almost  all  requisitions  are 
received  electronically  in  the  procurement 
office. 

In  other  agencies,  the  program  offices 
have  developed  automated  systems  to 
assist  in  their  budget  and  funds  control 
processes.  These  organizations  are  putting 
pressure  on  procurement  offices  to  accept 
their  requisitions  electronically.  There 
seems  little  point  to  the  user  in  printing 
the  data  from  their  electronic  system, 
placing  it  on  the  standard  agency 
requisition  form,  and  then  having  the  data 
re-entered  as  soon  as  it  enters  the 
procurement  office. 

The  key,  once  again,  to  maximize  the 
benefits  to  automation  throughout  an 
organization  is  interoperability: 
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agreement  on  a  set  of  data  items  and  their 
formats,  such  that  data  can  be  transferred 
from  one  system  to  another. 

INFORMATION  SOURCES 

All  procurement  offices  must  adhere 
closely  to  Federal  acquisition  regulations 
and  to  agency-specific  implementation  of 
these  regulations.  The  procurement 
process  is  complex,  however,  requiring 
access  to  procurement-specific  knowledge 
that  is  constantaly  changing. 

This  information  includes  the  regulatory 
framework  for  procurement  as  established 
in  the  FAR  and  its  applicable 
supplements;  availability  of  small  or 
disadvantaged  businesses  that  can  provide 
a  particular  commodity;  availability  of 
information  from  Government  mandatory 
sources  such  as  the  Federal  Prison 
Industries  (Ffl)  n»-  the  Committees  for 
Purchase  from  the  Blind  and  Other 
Severely  Handicapped;  availability  of  a 
particular  commodity  through  the  GSA 
Federal  Supply  schedules;  a  Wage 
Determination  from  the  Department  of 
Labor;  or  date  of  publication  of  a  synopsis 
is  the  Commerce  Business  Daily,  to  name 
a  few. 

These  information  sources  are  becoming 
available  in  electronic  forms.  A  small 
number  are  distributed  directly  from  the 
organizations  responsible  for  the  creation 
and  maintenance  of  the  information. 

However,  most  of  the  automated  tools 
available  are  through  private  companies, 
who  have  "reautomated"  the  information 
generated  by  the  Government. 

One  such  example,  is  the  Source  One 
product  available  on  CD-ROM  through 
Information  Handling  Services  (IHS). 


This  product  includes  the  products 
available  from  the  Federal  Prison 
Industries  (FPI),  the  Committee  for 
Purchase  from  the  Blind  and  Other 
Severely  Handicapped,  and  several 
hundred  firms  who  hold  Federal  Supply 
Schedules.  TTiis  product  has  been 
available  for  some  time,  originally 
beginning  as  a  microfilm  collection. 

Other  examples  include  the  text  of  the 
procurement  regulations  themselves: 
some  ten  companies  regularly  scan,  rekey, 
or  process  printer’s  tapes  of  the 
regulations,  and  provide  search  products 
on  CD-ROM  or  on  disk.  One  such 
product  is  Taurus,  a  CD-ROM  product 
produced  by  Government  Counselling,  Inc. 
which  contains  the  text  of  the  procurement 
regulations  (FAR,  FIRMR,  DFARS,  etc.) 
packaged  with  other  relevant  materials  for 
the  ADP  community  including  protest 
decisions  and  GSA/IRMS  publications. 

Both  of  these  examples  illustrate  the  need 
for  the  procurement  professional  to  access 
information  which  is  derived  from  a 
number  of  government  sources.  This  is 
illustrated  below: 
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Over  the  past  two  years  there  has  been 
some  recognition  on  the  part  of  the 
Federal  government  agencies  that  their 
responsibilities  go  beyond  creating  the 
information:  they  are  also  responsible  for 
distributing  it  to  their  users.  This 
responsibility  includes  distribution  through 
electronic  publishing.  The  most  significant 
of  these  electronic  publishing  initiatives 
has  been  made  by  GSA.  They  have  been 
issuing  a  CD-ROM  version  of  the  FAR, 
FIRMR,  and  related  publications  for  the 
past  few  year.*;.  The  product  is  distributed 
quarterly  through  GPO,  and  is  available 
for  an  annual  subscription  of  around  $120, 
making  it  a  viable  alternative  for  any 
Government  agency  or  contractor. 

CD-ROM  products  containing 
procurement  information  are  becoming 
common-place.  This  is  in  part  due  to  the 
acceptance  of  the  technology;  most 
procurement  offices  now  having  access  to 
a  CD-ROM  reader. 

Another  consideration  is  the  cost  of 
"mastering"  a  CD-ROM,  i.e.,  producing 
the  first  disk  for  duplication.  These  cost 
have  reduced  dramatically  over  the  last 
three  years.  It  is  now  possible  to  acquire 
equipment  that  will  master  a  CD-ROM  in 
your  own  office  for  less  than  $10,000. 
This  equipment  called  CD-R  (CD 
Recordable)  is  ideal  for  prototyping  new 
offerings  which  will  later  be  produced  in 
large  numbers  conventionally  or  for  the 
final  distribution  of  disks  which  have  a 
limited  distribution. 

This  ability  to  quickly  and  cheaply  create 
CD-ROMs  will  assist  Federal  agencies  to 
develop  procurement  information  sources 
which  are  specific  to  their  own 
environment  (e.g.,  contain  Federal 
regulations  as  well  as  department-wide 


regulations,  internal  policy  directives,  etc.), 
for  a  relatively  small  user  population. 

USING  ELECTRONIC  INFORMATION 
SOURCES 

The  electronic  sources  of  procurement 
information  which  are  available  today  are 
beginning  to  have  a  broad  base  of  users. 
The  sheer  volume  of  information  that  can 
be  held  on  a  CD-ROM  (equivalent  to  200 
Websters  dictionaries)  and  the  ability  to 
search  this  information  electronically  is 
significant. 

Three  issues,  however,  still  remain.  The 
first  of  these  relates  to  the  information 
needs  of  individual  organizations. 
Although  there  is  a  vast  amount  of 
procurement  information  pertaining  to  the 
Federal  government  as  a  whole,  each 
contracting  officer  (user)  needs  a  unique 
subset  of  this  information.  This  subset 
may  include  the  acquisition  regulations 
that  apply  to  the  user’s  organization  or  the 
information  sources  that  relate  only  to  the 
specific  commodity  that  is  being  acquired 
(e.g.,  ADP).  At  present,  this  information 
is  embedded  in  several  information 
sources  across  several  CD-ROMs,  often  by 
different  publishers. 

The  second  area  of  concern  is  the  many 
different  search  tools  which  must  be 
learned  to  use  the  various  CD-ROM 
products.  Different  publishers 
(Government  and  private)  have  used 
different  search  software  and  have 
organized  the  data  to  be  searched  in 
different  ways.  While  some  search 
systems  have  easy-to-use  help  functions,  it 
is  still  frustrating  for  users  to  move  from 
one  CD-ROM  to  another. 

The  final  area  of  concern  is  the  currency 
of  the  CD-ROM  disks.  Many  are  updated 
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quarterly  (e.g.,  GSA’s  CD-ROM  of  the 
FAR/FIRMR):  monthly  (e.g.,  IHS’s 
Source  One^  or  weekly  (e.g., 
Counterpoint’s  Federal  Register!  These 
updates  usually  are  not  frequent  enough 
to  ensure  the  data  on  the  CD-ROM  is 
current,  particularly  given  the  lag  time  to 
master  and  distribute  the  CD-ROM. 

Alternate  information  sources  must, 
therefore,  be  used  to  obtain  the  most 
current  information. 

BULLETIN  BOARDS 

Perhaps,  the  most  explosive  growth  in  the 
availability  of  electronic  procurement 
information  is  the  use  of  bulletin  boards. 

An  electronic  bulletin  board  is  a  central 
system  that  provides  dial-up  access  to 
users  through  electronic  communication. 
There  are  two  main  uses  of  procu.ement- 
related  bulletin  boards.  The  most 
prevalent  use  is  the  electronic  bid  board 
which  allows  users  to  identify  solicitations 
as  they  are  issued.  Some  bid  boards  also 
allow  firms  to  submit  their  bids 
electronically.  Traditionally  these  bid 
boards  are  used  to  publicize  contracting 
opportunities  under  small  purchase 
procedures. 

One  of  the  first  bulletin  boards  for  small 
government  procurement  was  established 
by  the  Naval  Supply  Center  in 
Jacksonville,  FL  in  1989.  The  system 
allows  subscribers  to  view  and  quote  on 
small  purchase  requirements.  The  system 
is  called  EASE  (Electronic  Assisted 
Solicitation  Exchange)  and  has  been 
extended  to  other  Naval  Supply  Centers. 

Also  of  significance  is  GATEC 
(Government  Acquisit'm  through 
Electronic  Commerce),  a  pilot  project  at 


the  Department  of  Air  Force,  Wright- 
Patterson  Air  Force  Base.  GATEC  is 
intended  to  be  a  DOD-wide  procurement 
bulletin  board,  with  a  single  organization 
(currently  Laurence  Livermore  National 
Laboratory  (LLNL))  acting  as  a  repository 
of  solicitations. 


Solicitations  are  posted  from  a  variety  of 
sites  including  Wright-Patterson  Air  Force 
Base  and  the  Defense  Personnel  Supply 
Center  in  Philadelphia.  Initial  capabilities 
include  providing  RFQs  and  receiving 
quotes  electronically.  Access  to  GATEC 
is  through  a  series  of  some  17  VANs,  each 
of  which  must  be  certified. 

A  number  of  government  organizations 
are  also  beginning  to  use  bulletin  boards 
for  larger  contracting  opportunities.  Since 
the  FAR  mandates  that  actions  over 
$25,000  are  published  in  the  Commerce 
Business  Daily,  these  organizations 
typically  advertise  the  requirement  with  a 
notice  such  as: 
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"Interested  vendors  may  access  the 
bulletin  board  to  retrieve  the 
solicitation." 

Organizations  with  recent  notices  in  the 
CBD  include  U.S.  Army  Information 
Systems,  Defense  Commercial 
Communications  Office  (DECCO),  and 
SPAWARS. 

The  second  use  of  procurement-related 
bulletin  boards  is  to  provide  access  to 
information  sources  where  data  is 
constantly  being  updated. 

For  example,  GSA  has  developed  bulletin 
boards  to  disseminate  changes  to  the  FAR 
in  both  proposed  and  final  form  shortly 
after  the  Federal  Acquisition  Circulars 
and  Proposed  Rules  are  published  in  the 
Federal  Register.  Changes  are  presented 
both  in  context  as  well  as  in  revised 
subsections,  sections,  and  clauses.  These 
FAR  bulletin  boards  are  accessible 
through  normal  E-MAIL  procedures  as 
part  of  the  FTS2000  telecommunication 
services  offered  by  AT&T  and  Sprint. 

A  second  information  source  which  is 
available  through  a  bulletin  board  is  the 
List  of  Parties  Excluded  from  Federal 
Procurement  or  Non-procurement 
Programs,  formerly  the  List  of  Debarred 
and  Suspended  Bidders. 

Another  GSA  innovation  is  the  Multi-Use 
File  for  Interagency  News  (MUFFIN) 
Gateway  which  allows  Federal  agencies  to 
review  electronic  catalogs  and  create 
simplified  purchase  order  using  model, 
part,  and  descriptive  information  from  the 
catalog. 

Once  again,  the  availability  of  these 
bulletin  boards  is  not  without  its  problems. 
There  are  currently  no  standards  covering 


when  and  how  a  government  agency 
establishes  a  bulletin  board. 

Many  agencies  have  chosen  the  lowest 
cost  solution;  a  small  microcomputer, 
with  a  single  modem,  and  commercially 
available  bulletin  board  software  (BBS), 
operating  at  their  office  location.  Other 
procurement  offices  have  selected  to  place 
their  procurement-related  information  on 
general  agency  or  command-wide  services. 
Finally,  most  VANs  provide  bulletin  board 
services. 

The  result  is  a  proliferation  of  bulletin 
boards.  To  make  use  of  them,  the  user 
must  first  know  that  they  exist,  obtain  the 
telephone  number  and  any 
telecommunications  protocols,  and  then 
dial-in  (after  long  distance).  Once  on  the 
bulletin  board,  the  user  is  faced  with  many 
different  menus  and  search  functions. 

ELECTRONIC  DISSEMINATION 

Electronic  sources  of  information  are  not 
only  a  problem  for  the  user: 
dissemination  of  information  through  CD- 
ROMs  or  electronic  bulletin  boards  can 
also  pose  problems  for  the  information 
source. 

Many  of  the  processes  which  generate  the 
procurement  information  to  be  the  subject 
of  the  dissemination  process  are 
themselves  automated.  Most,  however, 
use  some  form  of  in-house  or  proprietary 
wordprocessing  or  desktop  publishing 
software.  For  example,  a  Macintosh 
desktop  publishing  environment  is  used  by 
GSA  in  the  FAR  publication  process. 

These  systems  usually  use  proprietary 
formats  to  denote  the  special  formatting 
characteristics  of  a  piece  of  text  (e.g., 
bold,  underline,  tabs).  This  information  is 
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generally  lost  when  the  information  is 
transferred  to  the  systems  which  create  the 
CD-ROM  or  load  the  bulletin  boards. 
The  standard  method  for  transferring  such 
information  is  the  ASCII  file  which  does 
not  support  such  proprietary  coding 
schemes. 

Transfering  complex  information  from  one 
environment  to  another  can  be  costly  and 
time  consuming,  if  the  same  quality 
document  is  to  be  produced  electronically 
that  is  available  in  hard-copy.  Many 
information  sources  do  not  know  how  to 
do  this  efficiently. 

STANDARD  GENERALIZED  MARKUP 
LANGUAGE  (SGML) 

Standards  do  exist  to  facilitate  the  transfer 
of  text  from  one  publication  system  to 
another,  the  most  significant  of  these 
being  the  use  of  the  Standardized 
Generalized  Markup  Language  (SGML). 
SGML  is  a  widely  used  methodology  to 
standardize  the  way  publication-quality 
text  is  represented  that  permits 
interchange  between  different 
wordprocessing  or  publishing  systems. 
(Reference  3). 

SGML  uses  an  alternate  approach  to  the 
procedural  markup  used  in  conventional 
wordprocessing  systems.  A  word  or 
phrase  to  be  emphasized  as  a  certain 
"type"  (e.g.,  title)  but  not  to  specify  the 
type  of  emphasis  to  be  made  (e.g., 
boldface  italics).  This  is  called 
"descriptive  markup",  and  has  some  clear 
advantages. 

If  information  is  marked  as  "title"  rather 
than  a  sequence  of  words  marked  as 
"boldface  italics",  it  can  be  easily 
distinguished  from  the  rest  of  the 
document,  and  can  be  converted  to 


"boldface  italics"  when  printed  in  another 
system. 

Descriptive  markup  can  be  made  to  be 
independent  of  a  particular  product,  and 
can  be  designed  to  be  "read"  by 
computerized  systems  and  by  humans.  In 
this  way,  all  the  information  can  be  readily 
interchanged  not  just  the  text  itself. 

An  example  of  a  tagged  document  is 
shown  below: 


EXAMPLE  OF  TAGGED  DOCUMENT 
BEING  PARSED 

When  parsing  tins  document 

<'DOCTYPE  FAR  SYSTEM  r\MYDlR\FAR,D  1 1.'  • 
<Bo()k> 

<P>Finally  illusirations  and  nutlicinaiicai  tomiulac 
ina>  by  added  to  SGMl.  dociimcnis.</P> 

<XMP>Tcst  your  understanding 
<P>Try  to  anssser  the  follusvmg  questions  to  check 
whether  you  understood  the  material  presented  in  this 
chapter.  </P> 

</XMP> 

</Book> 

you  obtain  this  error  message 
Error  in  tag  close  at  >' 

XMP  end  tag  ignored;  doesn't  end  any  open  element  teurrent 
Book  I 


For  the  example  above,  a  format  error  has 
been  found. 

SGML  is  an  established  standard  for 
descriptive  markup.  It  is  a  set  of  rules  for 
descriptive  markup  and  provides  for: 

Defining  the  structure  of 
documents  and  the  logical  units  of 
which  it  is  composed;  and 

Tagging  individual  fields  with  text 
such  that  they  may  be  later 
processed  (e.g.,  in  print  preparation 
or  for  data  extraction  purposes). 
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Another  advantage  of  SGML  is  that  it  can 
be  used  to  ensure  that  text  provided  in  a 
prescribed  format  contains  certain  types  of 
data  or  is  in  a  particular  sequence. 

ELECTRONIC  DATA  INTERCHANGE 
(EDI) 

Electronic  Data  Interchange  (EDI)  is  the 
computer-to-computer  exchange  of  routine 
business  information  in  a  publicly 
accepted  standardized  format.  For  EDI  to 
work,  the  sender  and  receiver  of  the 
business  information  must  communicate  in 
the  same  format. 

The  computer-to-computer  interchange  of 
information  is  not  new.  Since  the  mid- 
1950s,  large  private  companies  have  been 
communicating  business  information 
electronically.  But,  because  users 
communicated  in  unique  formats, 
businesses  found  it  cumbersome  and  time 
consuming  to  expand  their  electronic 
communications  to  new  trading  partners. 

Today,  however,  nationally  and 
internationally  recognized  data  formats 
have  been  developed,  commonly  referred 
to  as  standards  or  transaction  sets  that 
allow  easy  interchange  of  data. 

The  Transportation  Data  Coordinating 
Committee  (TDCC),  formed  in  the  late 
1960s,  initially  created  EDI  standards  for 
rail,  motor,  air,  and  ocean  shipping.  Its 
success  at  developing  these  standards,  led 
other  industry  groups  such  as  grocery, 
chemical,  and  warehousing  to  seek  its 
help.  As  TDCC  created  industry-oriented 
standards,  some  companies  that  used  them 
saw  the  need  for  generic  standards  that 
cut  across  industry  boundaries. 

In  1979,  the  American  National  Standards 
Institute  (ANSI)  formed  the  Accredited 


Standards  Committee  (ASC)  X12  to 
develop  uniform  standards  for 
electronically  interchanging  business 
transactions  between  and  among 
industries.  The  ASC  X12  publishes  a 
common  data  dictionary  that  provides 
common  EDI  transactions.  Below  is  an 
example  of  an  ANSI  X12  data  segment 
definition.  (Reference  4). 


As  the  Federal  government  moves  towards 
electronic  commerce,  the  use  of  these  EDI 
standards  will  become  significant, 
(Reference  5). 

CONCLUSIONS 

More  and  more  information  sources  are 
available  in  electronic  form:  the  principle 
mechanisms  being  CD-ROM  electronic 
publishing  and  bulletin  boards. 

Individual  organizations  have  often 
introduced  these  products  to  fulfill  specific 
information  needs.  The  products 
generally  work  well  when  used 
individually. 
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However,  a  contracting  officer  needs 
access  to  many,  or  at  least  parts  of  many, 
information  sources.  Searching  across 
multiple  CD-ROM  sources,  or  accessing 
multiple  bulletin  boards  is  a  time 
consuming  and  frustrating  process 
requiring  high  levels  of  computer 
awareness. 

If  the  Procurement  Knowledge  Network 
(PKN)  is  to  work,  it  must  be  easier  for 
information  source  generators  to  publish 
their  materials  and  easier  for  the  field 
contracting  officer  to  use  them. 

This  transition  will  be  best  accomplished 
through  the  adoption  of  existing  standards, 
such  as  SGML  or  ANSI  X12. 
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PROTECTING  THE  U.S.  TECHNICAL  LEAD 


CPT  Robert  A.  Newton,  OASD(C3I) 


ABSTRACT 

Several  Department  of  Defense 
studies  have  indicated  that  acquisition 
programs,  which  were  designed  to 
achieve  a  combat  effective  lifetime  of  10 
to  15  years,  were  often  achieving  less 
than  a  third  of  this  lifetime.  In  fact,  50 
percent  of  the  evaluated  acquisition 
programs  had  countermeasures  deployed 
against  them  within  three  years  of  Initial 
Operational  Capability.  The  reason  for 
this  condition  was  the  compromise  or 
inadvertent  loss  of  information  to  foreign 
intelligence  services. 

Due  to  the  conditions  discovered 
by  these  studies,  the  Under  Secretary  of 
Defense  (Acquisition)  established  the 
Acquisition  Systems  Protection  Program 
(ASPP)  in  1991.  The  ASPP  concept 
required  program  managers  and  the 
acquisition  community  to  integrate 
security  planning  and  counterintelligence 
operations  directly  into  the  acquisition 
and  engineering  design  process. 

However,  the  goal  is  not  to 
protect  the  entire  system.  Rather,  the 
ASPP  concept  mandates  the 
identification  of  the  system’ s  most 
essential  elements  for  protection.  As  a 
result,  the  goal  is  to  increase  the 
protection  provided  to  these  key 
components,  while  reducing  the  overall 
cost  and  administrative  burden  of 
protecting  the  system  from  compromise. 

Introduction 

Although  the  direct  military 
threat  from  the  former  Soviet  Union  no 


longer  exists,  the  United  States  still  faces 
a  major  threat  to  its  national  security. 
This  threat  is  much  more  subtle  and 
diverse  than  the  overt  military  threat.  In 
this  case,  the  threat  is  the  effort  by  the 
states  of  the  former  Soviet  Union,  as 
well  as  other  nations,  to  acquire  and 
produce  key  technologies  developed  in 
the  United  States  for  military  or 
economic  advantages.  This  threat  is 
every  bit  as  grave  as  the  overt  military 
threat  from  the  Soviet  Union  used  to  be 
to  the  nations’s  security.  In  an  era  of 
multinational  corporations,  joint 
development  programs,  and  military 
grants,  the  loss  of  key  defense 
technology  is  likely  to  increase  unless 
positive  steps  are  taken  to  stem  the 
unauthorized  flow  of  information. 

Presently,  the  Department  of 
Defense  spends  about  eleven  billion 
dollars  a  year  on  research  and 
development  programs.  However,  unless 
we  can  ensure  these  programs  are  not 
compromised,  this  investment  will  be 
wasted.  The  purpose  of  this  research 
effort  is  to  ensure  our  soldiers  are  the 
best  equipped  and  most  confident 
soldiers  on  the  battlefield.  However,  if 
our  weapons  are  compromised;  their 
technology  copied  or  countered  by  the 
enemy  or  a  third  party,  then  our  soldiers 
will  not  have  this  advantage  during  the 
battle.  The  end  result  will  be  a 
significantly  higher  casualty  rate  for  our 
forces  than  our  commanders  should  have 
expected. 

Background 

Although  spies  and  espionage 
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have  existed  for  many  centuries,  the 
efforts  and  techniques  employed  by 
modem  intelligence  services  supplement 
these  techniques  with  sophisticated 
electronic  collection  systems.  However, 
the  greatest  threat  to  most  acquisition 
programs  is  still  the  human  agent.  In  this 
country,  the  spy’s  mission  is  aided 
immensely  by  our  open  society.  Many 
research  programs,  which  would  be 
classified  or  compartmentalized  in  other 
nations,  are  openly  discussed  in 
professional  journals  and  other  public 
forums  in  the  United  States. 

Although  the  United  States  had 
an  effective  system  to  counter  most 
intelligence  operations  of  traditional 
adversaries  during  the  Cold  War,  this 
system  will  face  a  new,  diversified  threat 
in  the  future.  In  addition  to  the  vast 
intelligence  collection  resources  of  the 
Russian  state,  many  of  the  Republics  are 
creating  their  own  intelligence  services. 
Further,  the  demise  of  the  Soviet  Union 
has  allowed  other  governments  to  divert 
intelligence  assets  to  the  collection  of 
technical  information  from  us.  As  the 
world  enters  the  new,  multi-polar  era, 
the  threat  of  industrial  and  economic 
spying  is  likely  to  increase  dramatically. 
This  trend  has  forced  the  United  States 
to  focus  new  attention  on  the  security  of 
its  research  and  development  efforts. 

Two  major  studies  clarified  the 
magnitude  of  the  threat  to  Defense 
acquisition  programs.  One  study  was 
conducted  under  the  direction  of 
Congress,  while  the  other  study  was 
directed  by  the  Under  Secretary  Of 
Defense  (Acquisition).  The  latter  study 
was  known  as  the  Protection  of  the  U.S. 
Technical  Lead  (PTL). 

Of  the  unclassified  findings,  the 


most  significant  ones  concerned  the 
management  of  security  programs  within 
the  acquisition  community.  The  first 
finding  noted  that  the  only  document 
which  addressed  security  issues  in  most 
collateral  programs  was  the  Security 
Classification  Guide.  However,  the 
finding  also  noted  that  the  guide  rarely 
contained  any  information  or  guidance 
on  the  scope  nor  intent  of  the  protection 
effort.  As  a  result,  most  programs  lacked 
a  coherent,  integrated  plan  to  protect 
the  technology  produced  within  the 
program. 

Similarly,  another  finding  noted 
that  DoD  lacked  central  direction  for 
protection  planning  and  oversight.  This 
condition  was  reflected  in  program 
offices  and  range  facilities,  where 
commanders  rarely  allocated  security 
programs  sufficient  resources  or 
personnel  to  implement  an  effective 
p.  Jtection  program.  Compounded  by  a 
lack  of  security  training  for  personnel  in 
the  acquisition  community,  these  findings 
underlined  the  need  for  a  "cultural" 
change  to  integrate  protection  planning 
into  the  protection  process. 

This  need  was  manifested  in  the 
study’s  recommendations.  The  most 
significant  recommendation  was  to  have 
USD(A)  assign  the  responsibility  for 
protection  planning  to  the  program 
managers.  Further,  the  study 
recommended  that  DoD  allocate  the 
counterintelligence  and  security 
specialists  to  the  program  offices  to  help 
the  program  managers  develop  an 
effective  protection  strategy.  In  addition, 
the  study  recommended  an  integration  of 
protection  planning  into  the  curriculum 
of  acquisition  training  programs.  Also, 
the  study  recommended  the 
establishment  of  a  DoD-level  office  to 
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act  as  the  focal  point  for  protection 
efforts  and  to  develop  a  Master  Plan  to 
address  infrastructure  issues. 

The  Congressional  study  was  the 
result  of  the  FY  91  and  FY  92  budget 
hearings.  After  a  review  of  selected 
range  facilities  and  selected  acquisition 
programs,  several  members  of  Congress 
became  extremely  concerned  about  the 
security  of  Defense  acquisition  programs. 
In  turn,  this  concern  resulted  in  several 
recommendations  from  Congress  to  the 
Secretary  of  Defense.  Many  of  the 
recommendations  were  similar  to  the 
conclusions  of  the  PTL  group.  In 
addition.  Congress  recommended  that 
DoD  to  establish  oversight  responsibility 
for  protection  planning  and  document 
the  cost  of  protection  measures.  Further, 
Congress  recommended  that  DoD 
develop  a  strategy  to  fix  the  security 
deficiencies  at  the  range  and  support 
facilities. 

These  findings  led  to  several 
changes  within  DoD.  One  of  the  most 
significant  changes  was  the  establishment 
of  the  Acquisition  Systems  Protection 
Office  (ASPO)  within  the  Office  of  the 
Secretary  of  Defense.  The  ASPO  was 
tasked  by  the  Under  Secretary  of 
Defense  (Acquisition)  to  (1)  review  the 
security  classification  guide  and  program 
protection  measures  of  all  major 
acquisition  programs,  and  (2)  provide  a 
written  assessment  of  these  documents  to 
the  Defense  Acquisition  Board 
Committee  prior  to  each  Milestone 
Review.  When  one  considers  the  number 
of  minor  acquisition  and  research 
programs  which  feed  the  major 
programs,  this  review  authority  is  more 
encompassing  than  it  may  appear. 

In  addition,  the  ASPO  was 


tasked  to  develop  a  plan  to  address  the 
security  infrastructure  problem  at  DoD 
facilities,  as  well  as  develop  a  plan  to 
ensure  the  uniformity  of  protection 
efforts  for  technology  or  information 
which  is  used  within  multiple  programs. 

The  other  major  change  was  the 
incorporation  of  protection  planning 
requirements  within  the  DoD  directives 
and  instructions  which  govern  acquisition 
management.  In  Part  5,  Section  F  of 
DoD  Instruction  5000.2,  program 
managers  are  told: 

"a  comprehensive  protection  and 
technology  control  plan  shall  be 
established  for  each  defense 
acquisition  program  to  identify 
and  protect  classified  and  other 
sensitive  information." 

Further,  this  same  section  states 
this  plan  will  address: 

•  the  use  of  counterintelligence  and 

operation  security  surveys  to 
monitor  information  loss  during 
system  development; 

•  the  definition  of  threat  options 

(reactive  threat)  and  the  potential 
for  exercising  those  options  which 
could  counter  the  acquired 
systems  capabilities; 

•  the  potential  vulnerabilities  of  the 

acquired  system  due  to  evolving 
threat  capabilities;  and 

•  for  international  programs, 

technology  assessment  and  control 
will  be  addressed. 

With  these  initiatives,  DoD  has 
taken  action  to  address  the  concerns 
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expressed  in  these  studies.  However,  the 
Department  faced  the  equally  tough  task 
of  bringing  the  10(X)(+)  acquisition 
programs  into  compliance  with  these 
guidelines.  The  primary  method  DoD 
has  elected  to  use  to  fulfill  its  oversight 
responsibility  is  through  the  review  of 
each  system’s  or  component’s  Program 
Protection  Plan  as  part  of  the  Defense 
Acquisition  Board  review  process. 

THEPPP 

Since  the  development  of  a 
program  protection  plan  is  a 
requirement  for  all  acquisition  programs, 
acquisition  personnel  must  be  familiar 
with  the  goal  and  objectives  of  the  plan. 
The  ideal  plan  is  one  which  blends 
elements  of  counterintelligence  analysis, 
operations  security,  traditional  security 
disciplines,  and  system  security 
engineering  to  provide  an  efficient  and 
cost-effective  strategy  which  will  protect 
the  system  from  all  collection  threats 
during  development  and  deployment.  To 
ensure  the  protection  plan  is  fully 
integrated  into  the  program’s  operations, 
DoD  Instruction  5000.2,  Part  5,  Section 
F  directs  program  managers  to; 

•  develop  a  protection  plan  prior  to 
the  Milestone  I  Review  and 
update  it  prior  to  each  subsequent 
review; 

•  produce  a  protection  plan  that 
.will  include  program  related 
activities  at  test  centers,  ranges, 
laboratories,  contractor  facilities, 
and  deployment  locations  to 
provide  protective  measures  for 
all  phases  and  aspects  of  the 
acquisition  process. 

Presently,  DoD  is  using  an 


assessment  of  the  Program  Protection 
Plans  of  all  major  acquisition  programs 
(and  their  component  systems)  to 
measure  the  degree  of  compliance.  The 
ASPO  has  been  directed  to  provide  a 
written  report  to  the  DAB  Committee 
on  the  effectiveness  of  the  plan  prior  to 
each  review.  When  programs  are  found 
to  be  deficient,  the  DAB  Committees 
have  directed  the  programs  to  revise 
their  plans  and  resubmit  them  at  a  later 
date. 

With  this  degree  of  emphasis  on 
the  PPP  in  the  DAB  review  process,  one 
may  wonder  what  criteria  the  ASPO  will 
use  to  review  the  PPPs.  The  Program 
Protection  Plan  should  accomplish  two 
goals  for  the  program.  First,  it  should 
serve  as  a  unified,  integrated  strategy  for 
protecting  the  key  technologies  and 
information  of  the  system  during  all 
phases  of  development.  Second,  the  plan 
must  clearly  and  concisely  identify  the 
system’s  Essential  Information, 
Technologies,  or  Systems  (EPITS)  for 
protection  and  formally  commit  to  a 
strategy  to  ensure  their  protection  from 
compromise  or  unauthorized  disclosure. 

The  ASPO  evaluates  the 
protection  plans  based  upon  a  series  of 
exit  criteria.  The  first  element  of  the 
plan  which  comes  under  scrutiny  is  the 
system  description  and  program 
description.  The  system  description  must 
clearly  describe  the  mission,  military 
value,  and  operational  parameters,  as 
well  as  identifying  any  supported  or 
supporting  programs.  The  program 
description  should  discuss  the 
organization  of  the  program  office  and 
identify  the  locations  of  any  facilities 
where  essential  program  elements  will  be 
stored, tested,  analyzed,  or  produced. 
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Perhaps  the  most  critical  element 
of  the  review  is  the  analysis  of  the 
system’s  Essential  Program  Information, 
Technologies,  and  Systems  (EPITS).  The 
EPITS  are  those  elements  of  the 
program  which,  if  compromised,  would 
cause  a  reduction  in  combat 
effectiveness  or  reduce  the  combat 
effective  lifetime.  The  EPITS  are  those 
elements  which  give  the  system  its 
unique  ability  on  the  battlefield.  The 
identification  of  the  EPITS  in  the 
protection  plan  plays  the  same  role  as 
the  commander’s  intent  in  an  operations 
order;  they  focus  the  protection  efforts 
and  clearly  delineate  what  elements  must 
be  protected  to  ensure  mission 
accomplishment.  By  providing  this 
focus, the  acquisition  community  should 
be  able  to  reduce  the  costs  of  protection 
while  increasing  the  level  of  security 
provided  to  the  program. 

To  properly  identify  the  EPITS, 
the  program  manager  must  "decompose" 
the  system.  In  effect,  the  program 
manager  needs  to  identify  the  specific 
component  or  system  which  allow  the 
weapon  to  accomplish  each  of  its 
assigned  functions  on  the  battlefield. 
Subsequently,  the  program  manager 
determines  if  the  identified  element  is  an 
EPITS  by  answering  four  questions: 

If  a  foreign  intelligence  service  obtain 
this  information  or  item, 

•  could  they  devise  a  method  to  1^ 
my  system? 

•  could  they  devise  a  method  to 
clone  my  system? 

•  could  they  devise  a  method  to 
neutralize  my  system? 


•  would  I  have  to  change  the  design 

to  ensure  the  same  level  of 

superiority  on  the  battlefield? 

If  the  answer  to  any  of  these 
questions  is  "Yes", the  item  would  qualify 
as  a  possible  EPITS.  Closely  related  to 
these  questions  is  the  requirement  to 
define  the  loss  criteria.  TTie  program 
manager  must  determine  if  the  mere 
possession  of  the  item,  knowledge  about 
its  abilities,  or  the  ability  to  reproduce 
the  item  constitutes  loss.  This  definition 
will  play  a  critical  role  in  the  evaluation 
of  the  collection  threat  to  the  program. 

Once  the  initial  screening  of  the 
EPITS  has  been  accomplished,  a 
refinement  process  needs  to  occur. 
Specific  components  or  pieces  of  high 
technology  should  be  consolidated  under 
common  sub-systems.  These  specific 
elements  are  known  as  Sub-systems  or 
Technologies  (SOTs).  In  addition,  the 
program  officials  should  attempt  to 
define  the  elements  as  closely  as  possible 
to  recognized,  DoD  terms  based  upon 
lists  such  as  the  DoD  Key  Technologies 
List  or  the  National  Disclosure  Policy 
categories.  This  consolidation  and 
refinement  will  assist  the  intelligence 
analyst  identify  the  collection  threat  to 
the  system. 

One  of  the  major  tasks  for  the 
protection  plan  is  its  role  as  a 
counterintelligence  document.  By  tasking 
the  intelligence  community  to  provide 
the  information,  the  program  manager 
identifies  the  existing  and  anticipated 
intelligence  collection  threat  to  the 
program.  The  program  manager  should 
task  the  intelligence  community  to 
identify  (1)  which  countries  have  an 
interest  in  the  system’s  EPITS,  (2)  what 
level  of  research  those  countries  are 
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conducting  in  these  areas,  and  (3)  what 
capability  those  countries  have  to  collect 
information  on  the  EPITS.  When  the 
answers  to  these  questions  are  matched 
to  the  locations  where  EPITS-related 
information  exists,  the  identification  of 
the  program’s  vulnerabilities  becomes 
possible. 

Once  the  vulnerabilities  are 
identified,  the  countermeasures  program 
can  be  designed.  Countermeasures 
should  only  be  implemented  if  a 
vulnerability  exists,  due  to  regulatory 
requirements,  or  due  to  the  use  of 
classified  or  sensitive  EPITS  material 
from  another  program.  If  one  of  these 
conditions  does  not  exist,  the  program 
manager  should  not  waste  limited 
resources  protecting  the  system  from  a 
threat  which  does  not  exist.  However, 
the  program  manager  must  realize  that 
the  threat  environment  is  dynamic  and 
new  conditions  may  develop  which  will 
require  a  change  of  the  protection 
concept  to  protect  the  system’s  EPITS. 

Once  the  vulnerabilities  and  other 
protection  requirements  are  identified, 
the  program  manager  should  place  them 
in  a  priority  sequence  based  upon  their 
impact  to  the  program  if  compromised. 
Subsequently,  an  informal,  cost-benefit 
analysis  should  be  performed  to 
determine  which  elements  can  be 
protected  with  the  available  resources. 
Ideally,  all  vulnerabilities  concerning 
EPITS  should  be  countered.  Once  this 
data  is  compiled,  the  countermeasures 
and  cost  data  should  be  included  in  the 
protection  plan. 

Three  enclosures  supplement  the 
protection  plan.  The  time  or  event- 
phased,  Security  Classification  Guide 
provides  guidance  to  all  users  or 


producers  of  the  system  on  the  level  of 
protection  which  should  be  supplied  to 
various  components.  As  such,  it  should 
be  oriented  upon  the  system’s  EPITS. 
The  Guide  should  not  only  specify  the 
classification  level  of  material,  but  it 
should  indicate  which  material  should  be 
protectively  marked  or  limited  in 
distribution  in  accordance  with  DoD 
5200.  IR.  Also,  the  Guide  should  contain 
specific  criteria  for  the  reduction  of 
classification  requirements  or  the 
termination  of  protective  restrictions. 

The  Technology  Assessment  / 
Control  Plan  is  used  to  assess  the  risks 
and  benefits  of  joint  development 
programs  with  foreign  countries  or 
foreign  military  sales.  In  addition,  if  such 
arrangements  are  approved,  the  plan 
provides  the  specific  guidance  required 
to  protect  essential  U.S.  technology  from 
compromise 

The  System  Security  Annex 
addresses  the  security  threats  and 
vulnerabilities  of  the  system  in  the 
operational  environment.  The  annex  is 
an  engineering  plan  which  discusses  how 
the  system’s  design  will  be  modified  to 
reduce  security  costs  and  burdens  upon 
the  user  after  deployment.  In  addition, 
this  annex  will  discuss  how  the  existing 
design  could  be  modified  to  allow  the 
sale  or  production  of  the  system  abroad. 

The  complete  Program  Protection 
Plan  (with  annexes)  provides  a  complete, 
integrated  program  which  protects  the 
system  from  the  establishment  of 
Mission  Need  until  the  system  is  retired 
from  service.  The  Program  Protection 
Plan  is  the  only  plan  in  the  acquisition 
process  which  addresses  security  issues. 
As  such,  it  is  an  extremely  valuable 
document  for  the  acquisition  community, 
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the  users,  support  facility  commanders, 
and  oversight  officials. 

Why  Should  I  Care? 

Many  individuals  within  the 
acquisition  and  intelligence  communities 
question  the  need  for  these  changes  by 
DoD.  Several  individuals  view  this 
requirement  as  nothing  more  than 
another  "hurdle"  which  program 
managers  must  "jump"  over.  These  views 
are  wrong!  The  Acquisition  Systems 
Protection  Program  is  a  direct  result  of 
recognized  problems  associated  with  the 
protection  of  weapons  programs  and  key 
technologies  from  compromise  or  loss. 
Failure  to  protect  the  system  from 
compromise  can  only  lead  to  increased 
battlefield  casualties  should  U.S  forces 
have  to  use  the  system  in  combat. 

Another  complaint  is  this  new 
concept  is  not  any  different  from  the 
system  of  regulations  and  directives 
which  already  existed.  Once  again,  this 
view  is  not  in  focus.  Although  the 
concept  relies  upon  many  of  the  same 
regulations  and  policies  which  existed  in 
the  past  for  guidance,  the  true  difference 
is  the  scope  and  range  of  the  ASP 
program.  The  ASP  concept  requires  the 
program  managers  to  identify  the  most 
essential  elements  of  the  program  for 
protection,  not  the  entire  system.  Once 
these  elements  are  identified,  the 
manager  must  task  the  intelligence 
community  to  identify  which  foreign 
intelligence  services  have  the  interest 
and  capability  to  collect  the  information 
instead  of  relying  on  a  "worst-case" 
scenario.  If  a  threat  or  vulnerability  does 
not  exist,  countermeasures  do  not  have 
to  be  developed.  The  countermeasures 
program  must  provide  specific  guidance 
on  the  protection  strategy  to  all 


personnel  who  have  EPITS  information 
within  their  possession.  As  a  result,  we 
are  shifting  the  emphasis  of  protection 
efforts  from  "regulatory  compliance"  into 
an  era  where  the  program  manager  is 
required  to  outline  a  specific  protection 
strategy  and  providing  the  needed 
direction  to  counter  known  or 
anticipated  threats  to  the  system.  This 
focus  allows  DoD  to  reduce  significantly 
the  amount  of  material  which  must  be 
classified,  while  concentrating  the 
protection  efforts  on  the  most  important 
elements  of  the  program. 

Another  common  misconception 
is  that  the  ASP  program  will  increase 
program  costs.  The  most  common  reason 
for  this  belief  is  that  the  ASP  program  is 
just  a  new  version  of  Special  Access 
Programs  (SAP).  Unlike  SAPs,  the  ASP 
concept  does  not  orient  on  protecting 
everything  (including  the  program’s 
name)  from  compromise  or  loss.  The 
goal  of  ASP  is  to  protect  the  10-15%  of 
the  program  which  gives  the  system  its 
unique  combat  capability.  As  a  result, 
protection  of  these  key  elements  can  be 
significantly  increased  while  maintaining 
or  reducing  the  cost  of  protection. 

Conclusion 

The  Acquisition  Systems 
Protection  Program  is  a  major  effort  by 
DoD  to  protect  the  technical  advantage 
that  American  soldiers  have  traditionally 
enjoyed  on  the  battlefield.  Unless  U.S. 
technology  is  protected  and  weapons  are 
not  compromised,  the  likelihood  of 
increased  battlefield  casualties  is 
significant.  The  ASP  concept  is  designed 
to  increase  the  protection  provided  to 
the  most  essential  elements  of  Defense 
research  while  reducing  the  cost  and 
administrative  burden  of  protection 
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CPT  Newton  is  assigned  to  the  Office  Of 
the  Assistant  Secretary  of  Defense 
(Command,  Control,  Communication  and 
Intelligence).  He  is  an  acquisition  officer 
who  earned  his  undergraduate  degree  in 
1981  from  the  U.S.  Military  Academy  and 
his  Master’s  degree  from  the  University  of 
Virginia.  He  has  served  with  tactical  units 
in  Korea  and  at  Fort  Hood  and  he  has 
served  as  an  Assistaiu  Professor  at  the 
U.S.  Military  Academy.  Upon  the 
completion  of  his  tour  at  OSD,  he  will 
proceed  to  the  Command  And  General 
Staff  College. 
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The  Users’  Stake 


In  the  Automation  of  Government  Contract  Offices 

Lawrence  H.  Oppenheimer,  Price  Waterhouse 


ABSTRACT 

In  the  last  few  years,  federal  contracts 
offices  have  begun  to  automate  their 
financial  and  administrative  functions. 
These  efforts  have  focused  either  on 
specific  functions  within  the  offices,  or  on 
the  entire  operations  of  the  office.  As 
government  managers  examine  alternative 
approaches  toward  automation,  and 
alternative  office  processes  after 
automation,  contracts  professionals  within 
the  organizations  should  recognize  that 
they  have  a  lot  at  stake. 

The  objectives  of  automation,  and  how 
automation  is  implemented,  is  critically 
important  to  the  people  within  a  contracts 
office.  Although  more  automation  is 
inevitable,  the  record  of  successes  to  date 
has  not  been  uniform.  The  purpose  of  this 
paper  is  to  identify  issues  in  automation, 
how  different  approaches  can  have 
unanticipated  effects  on  the  professionals 
in  the  offices,  and  how  management  and 
the  users  can  maximize  some  of  the 
positive  (and  avoid  some  of  the  negative) 
effects  from  automation. 

INTRODUCTION 

This  paper  is  intended  to  be  a  "consumer’s 
guide"  to  automation  of  Federal  Govern¬ 
ment  contracts  offices.  Although  the  focus 
is  only  on  the  contracting  operations,  some 


of  the  principles  can  be  applied  to  other 
functional  office  environments.  In  fact, 
most  of  the  principles  were  originally 
developed  through  automation  of  other 
types  of  operations. 

The  paper’s  provides  a  very  brief 
description  of  the  automated  tools  that  are 
currently  being  implemented,  and  the 
benefits/objectives  of  management’s 
investment  in  automation.  Then  the  paper 
describes  the  prospective  effects  of 
automation  on  office  performance,  office 
work  patterns,  and  the  people  working  in 
the  office. 

The  paper’s  focus  is  on  the  changes  that 
the  current  contracts  professionals  are 
likely  to  experience,  and  the  actions  that 
people  can  take  to  improve  the  likelihood 
that  automation  will  improve  office  perfor¬ 
mance,  the  office  environment,  and  profes¬ 
sional  opportunities. 

A  few  last  notes:  This  paper  is  concerned 
with  offices  that  do  "Contracting"  versus 
"Purchasing",  -  meaning  that  Contracting 
offices  are  those  that  do  procurements  over 
$25,000.  Automation  of  purchasing  offices 
is  importantly  different  from  that  of 
contracting  offices,  and  is  at  this  point 
more  advanced. 

Second,  by  automation,  we  do  not  mean 
word-processing,  or  use  of  shrink-wrapped 


or  in-house-developed  applications  on 
commercial  software.  Instead,  we  mean  the 
use  of  specially -developed  and  customized 
commercial  cools  for  contract  operations. 
Also,  the  emphasis  is  on  the  effects  on 
career  contracts  professionals,  not  on  either 
technical  ADP  people  or  on  temporarily 
attached  people. 

1.  What  Automated  Tools  are  now 
Available  for  the  Federal  Contract 
Environment? 

In  recent  years,  a  large  number  of  new 
software  tools  have  been  specially 
developed  for  federal  contract  operations. 
The  new  tools  include  comprehensive 
systems  designed  to  meet  and  integrate 
most  of  the  requirements  of  a  Contracts 
Office,  and  also  specific  special  purpose 
contracts  tools.  These  new  systems  and 
tools  generally  perform  much  better  than 
the  systems  that  were  introduced  five  to 
ten  years  ago,  and  are  starting  to  win 
acceptance  among  users. 

For  the  most  part,  these  new  tools  are 
designed  to  take  advantage  of  the  faster 
and  more-powerful  micro-processors  now 
available  (386  and  486),  use  the  newer 
operating  systems  (UNIX  and  Windows), 
and  are  written  in  modern  languages  (such 
as  the  relational  database  management 
systems). 

Some  contracts  offices,  particularly  in  the 
DoD,  have  installed  and  implemented 
various  flavors  of  sophisticated  and 
comprehensive  contract  management 
systems.  Many  offices,  both  in  DoD  and 
outside  of  DoD,  have  purchased  special- 
purpose  tools  to  improve  the  operations  of 
certain  office  functions.  These  tools 
include: 


•  Management  Aids  and  Tracking 
Tools 

•  Form  Generators 

•  Data  Retrieval  Tools 

(FAR,  DFARS,  FIRMR,  etc.) 
(Vendor  Information) 

•  Document  Assembly  Tools 

(Solicitations,  Contracts,  SOWs) 

•  Automatic  Agreement  Generators 

•  Communications  (EDI,  etc.) 

•  Financial  and  Contract  Accounting 

•  Asset  and  Property  Tracking 

The  purpose  of  this  paper  is  not  to  analyze 
or  evaluate  the  specific  tools  or  systems, 
but  instead  to  show  that  the  work  of  the 
professionals  in  the  contracts  offices  is 
affected  by  the  success  or  failure  of 
automation,  and  that  those  professionals 
can  learn  from  the  experience  of  others  and 
can  take  measures  to  enable  automation  to 
achieve  its  objectives. 

2.  Objectives  of  Automation 

In  many  cases,  management’s  primary  ob¬ 
jectives  of  automation  are  to  reduce  costs, 
and/or  to  improve  efficiency.  They  intend, 
through  implementation  of  a  system  or  a 
tool,  to  reduce  the  labor  involved  in 
various  contract  functions. 

From  the  view  of  existing  employees  of  a 
contracts  office,  it  is  critically  important 
that  automation  must  result  in  a  substantial 
reduction  of  manual  labor  to  perform  a 
function  or  functions,  otherwise  the  staff's 
workload  will  become  heavier  and  less- 
fulfilling. 

The  next  objectives  cited  by  management 
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are  to  either  improve  the  quality  of  the 
organization’s  output,  or  to  enable  the  staff 
to  accomplish  a  function  that  they  could 
not  reasonably  accomplish  before  -  such  as 
to  communicate  by  EDI. 

Quality  and  internal  consistency  are 
common  justifications  for  automatic  soli¬ 
citation  software  and  automatic  contract 
generators. 

Similarly,  management  often  seeks  to  use 
custom  applications  of  accounting  software 
to  organize  and  report  on  contract 
financials.  Automatic  systems  are  often 
implemented  to  improve  the  integrity, 
accuracy,  and  storage  of  contract  data. 

Another  objective  cited  by  management  is 
to  use  program  management  and  tracking 
tools  in  order  to  improve  internal  control 
and  oversight  by  management.  Frequently, 
the  workload  in  contracts  offices  is 
difficult  to  measure  and  balance.  Some 
DoD  offices  have  stolen  a  technique  from 
defense  contractors  and  have  implemented 
internal  cost  control  systems. 

3.  The  Effects  of  Automation  on 
Performance 

In  these  days  of  cost-cutting  and  cost 
analysis,  it  is  important,  and  sometimes 
illuminating,  to  examine  the  results  of 
automation  on  contract  offices  in  order  to 
determine  whether  the  investment  achieved 
the  original  objectives. 

In  most  cases,  it  appears  to  this  writer  that 
functional  effort  is  saved  -  often 
substantially.  Total  workload,  however,  is 
NOT  reduced,  simply  because  the  overall 
efforts  are  redistributed  among  functions. 


In  most  cases,  this  redistribution  of  effort 
has  positive  results.  The  staff  is  able  to 
concentrate  efforts  on  functions  that  they 
were  not  able  to  do  before.  In  other 
words,  the  quality  of  output  improves. 

In  some  cases,  it  is  not  clear  that 
automation  accomplished  savings  in  the 
workload  (although  different  analytical 
approaches  can  come  to  different 
conclusions).  If  efficiency  is  an  objective, 
and  gains  in  efficiency  are  not  achieved, 
then  the  automation  project  failed  in  at 
least  that  respect.  When  this  happens,  to 
either  a  greater  or  lesser  degree,  then  the 
existing  professionals  in  the  office  suffer, 
because  they  are  usually  required  to  do 
more  work,  in  order  to  "serve  the 
computer"  and  accomplish  their  principal 
work. 

An  important  fact  to  note  is  that  the  overall 
results  of  automation  are  very  difficult  to 
predict.  Often  management  is  surprised  - 
and  frequently  disappointed.  And  the 
people  working  in  the  office  are  often  very 
surprised  at  the  way  their  work  has 
changed. 

However,  office  functions  of  various  types 
have  been  altered  by  automation  for  over 
twenty  years,  and  some  lessons  can  be 
applied  to  contracts  offices  in  order  to 
make  the  outcomes  more  predictable. 
Generally,  the  effects  of  automation 
depend  on: 

A.  The  obiectiveCs)  to  be  achieved. 
Whether  cost  or  staff  reductions  are  the 
only  objective,  or  one  of  several  objectives. 

B.  The  means  selected.  Whether  the 
approach  is  to  automate  an  existing  manual 
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function,  without  any  changes  in  processes 
or  responsibilities,  or  to  use  automation  in 
concert  with  a  planned  and  coordinated 
change  in  the  office’s  processes.  Often, 
the  result  of  automation  without  a  process 
change  is  disappointing.  It  requires 
substantial  managerial  effort  and  planning 
to  effect  process  change  through 
automation,  but  the  results  are  usually 
positive.  The  means  can  be  evolutionary  - 
such  as  through  a  TQM  process  -  or 
revolutionary  -  such  as  through  Business 
Process  Transformation. 

C.  The  emphasis  placed  on  user  input  and 
involvement.  It  is  fundamental  that  users 
be  a  part  of  system  design  and  implemen¬ 
tation,  and  that  they  must  be  a  part  of  the 
solution.  Users  must  "buy  into  a  project", 
for  it  to  be  acceptable  to  them.  This  is 
obvious  when  existing  processes  are  being 
automated.  It  is  even  more  important,  and 
more  difficult,  to  get  "buy-in"  when  the 
processes  themselves  are  being  changed. 

D.  The  success  of  the  implementation 
phase.  When  the  software  is  ready  for 
installation,  the  software  developer  or 
vendor’s  objective  is  to  complete  accep¬ 
tance  testing  and  to  receive  sign-off.  It  is 
difficult  for  either  party  to  concentrate  on 
user  training,  data  conversion  and  data 
quality,  and  to  accomplish  the  required 
adaptations  and  alterations  needed  for  a 
smooth  transition. 

E.  Training.  Whether  or  not  the  users  are 
fully  trained  in  the  delivered  software. 
This  includes  more  than  functional  and 
technical  training.  It  includes  post-cutover 
training  in  the  details  and  capabilities  of 
the  software. 

F.  Support  during  and  after  transition. 


Whether  the  software  developers  provide 
on-site  support  to  the  users  of  the  new 
software  during  the  transition  phase  after 
acceptance,  but  prior  to  system  main¬ 
tenance. 

Many  successful  system  developers  use  a 
process  of  "pro-active  technical  support"  to 
provide  technical  services  to  users  after 
cutover  and  initial  use  of  the  system.  The 
theory  is  that  users  develop  skills  with 
software  systems  over  a  period  of  several 
months  after  cutover.  It  is  during  that 
period  that  on-site  t-^chnical  assistance  is 
necessary  and  useful.  Pro-active  support 
often  includes  refresher  training,  advanced 
techniques,  periodic  user  group  meetings, 
and  "feedback"  meetings. 

G.  Motivation  of  the  Users.  If  the  users 
want  the  system,  are  involved  in  it,  and 
believe  that  they  have  a  stake  in  its 
successful  implementation,  then  they  will 
be  champions  of  it. 

One  of  the  features  most  often  requested 
by  management,  but  less  often  requested 
by  users,  is  system  documentation. 
Although  adequate  documentation  is 
necessary,  especially  for  technical 
reference,  most  users  depend  on 
documentation  less  than  management 
would  expect.  Quality  documentation  must 
be  provided  in  all  cases;  however,  it  is 
insufficient  by  itself  to  result  in  success. 

The  end  result  is  that  in  a  number  of  cases, 
the  original  objectives  are  not  attained,  and 
the  benefits  of  the  investment  are  not 
realized.  This  can  result  in  a  loss  of 
investment,  a  loss  of  time  by  users,  or  the 
use  and  reliance  on  a  "sick"  or  incom¬ 
pletely  implemented  system  for  a  long 
period  of  time. 
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Unfortunately,  anecdotal  evidence  indicates 
that  under-used  and  abandoned  tools  and 
automated  systems  are  not  uncommon  in 
the  contracts  field.  One  vendor  of  a  FAR 
retrieval  system  said  that  her  product  was 
the  third  such  service  purchased  for  use  in 
one  small  contracts  office. 

The  problem  with  automation  of  contracts 
office  functions,  and  the  reason  why  many 
projects  are  disappointments  is  that 
automation  of  a  contracts  office  is  an 
automation  of  a  process.  This  is 
importantly  unlike  the  automation  of  data. 
which  is  what  is  done  when  an  accounting 
operation  is  automated.  The  difference  is 
fundamental.  We  are  usually  accustomed 
to  thinking  in  terms  of  automating 
accounting  or  personnel  functions,  where 
the  system  addresses  data  and  records 
about  data.  The  processes  used  by  the  data 
is  usually  relatively  straight-forward. 

In  a  contracts  office,  automation  usually 
addresses  processes,  and  usually  there 
exists  a  number  of  ways  to  accomplish  a 
function  -  all  but  one  of  which  are  not  in 
current  use.  Use  of  the  computer  to  do  a 
process  requires  a  change  in  habits,  even  if 
the  functional  process  remains  the  same. 
If  the  functional  process  itself  changes,  as 
well  as  the  procedure  to  use  the  computer, 
then  users  often  elect  not  to  use  the 
automated  system  unless  the  immediate 
benefits  are  very  high.  As  a  result, 
contracts  professionals  face  some  extra 
risks,  and  have  a  great  deal  at  stake  when 
they  automate. 

4.  The  Effects  on  People 

This  analysis  does  not  address  the  technical 
support  people  who  service  and  maintain 
automated  systems.  Clearly,  automation 


will  require  technical  people  to  manage  the 
software. 

As  a  side  note,  the  idea  that  an  office  can 
develop  its  own  people  to  maintain  a 
system  is  one  of  those  myths  that  bedevil 
managers  and  cause  significant  suboptimal 
use  of  systems  and  tools.  Maybe  ten  years 
ago,  it  was  possible  for  certain  motivated 
people  to  leam  the  innards  of  a  system  so 
well  that  they  could  eventually  maintain  it. 
But  with  today’s  sophisticated  systems, 
that  idea  is  a  dangerous  dream.  When 
management  installs  a  system,  or  an  auto¬ 
mated  tool,  it  should  recognize  that  it  is 
also  buying  technical  support  services. 

The  present  employees  of  a  contracts  office 
have  an  important  stake  in  the  way 
automation  is  accomplished,  and  in  the  end 
result  of  automation.  They  will  be  directly 
affected  with  respect  to: 

•  The  way  they  will  spend  their  time 
in  the  future  office 

•  Their  professional  development 

•  Opportunities  for  advancement 

•  Their  income 

•  Job  satisfaction 

Although  the  end  users  are  often  passive 
observers,  they  should  understand  that  they 
frequently  have  more  at  risk  than 
management  or  the  vendor.  At  some  point, 
both  management  and  the  vendors  have  an 
interest  in  "declaring  victory"  and  moving 
on.  But  the  contract  professionals  have  to 
live  with  the  delivered  system  and  make 
the  best  of  it,  or  quietly  bury  it.  They  are 
the  ones  who  can  tell  you  whether  the 
system  serves  them,  or  whether  they  serv'e 
the  system. 
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Changes  and  Disputes 


If  the  system  serves  the  objectives,  then 
there  will  be  more  time  for  people  to  think, 
rather  than  simply  function.  For  example, 
if  a  system  is  installed,  there  should  be: 

•  Less  manual  clerical  effort,  and  less 
need  for  secretarial  support 

•  More  people  spending  a  majority  of 
their  time  on  Pcs,  rather  than 
writing  at  desks 

•  More  sharing  of  information,  and 
use  of  common  (internally 
consistent)  information 

•  More  reliance  on  the  system,  so 
that  system  upsets  become  major 
office  upsets 

•  More  time  on  exception  processing, 
less  on  routine 

The  opportunity  also  exists  for  people’s 
functional  skills  to  change,  and  for  the 
requirements  for  skills  to  change  in  a 
significant  way. 

If  automation  is  successful,  then  there  will 
be  less  time  in  clerical  functions,  there  will 
be  more  time  for  analysis,  and  for  use  of 
professional  skills.  There  will  be  more 
time  to  choose  among  alternatives,  and 
more  pressure  to  make  the  right  choice, 
and  to  properly  analyze  and  document  the 
choice. 

As  examples,  one  should  probably  expect 
increased  demand  for  the  following 
contract-related  skill  areas  in  procurement 
offices: 

•  Business  Research  (New  Vendors) 

•  Economics  (Cost-Price  Analysis) 

•  Subcontracts 


5.  Lessons  Learned 

The  record  of  automation,  like  most 
records,  is  that  some  projects  succeed,  and 
some  fail.  The  bad  news  is  that  the 
projects  in  contracts  offices  that  fail,  or 
that  fall  short  of  success,  are  discour- 
agingly  numerous. 

The  good  news  is  that  it  is  not  impossibly 
difficult  to  predict  failures,  or  to  identify 
troubled  projects  early  in  the  development 
and  implementation  phases.  Mid-course 
corrections  CAN  rescue  troubled  projects. 
And  more  importantly,  early  planning  and 
management  follow-through  can  greatly 
reduce  the  risks  of  failure. 

Contracts  professionals  should  be  alert,  and 
be  involved  in  decisions  about  automation, 
because  they  have  so  much  at  stake,  and 
because  their  involvement  can  be  crucial  to 
success. 

The  themes  of  this  "lessons  learned" 
section  are  that:  (1)  contracts  professionals 
can  affect  the  probability  that  their  projects 
will  succeed,  (2)  the  purchase  price  of  the 
software  itself  is  just  one  component  of  the 
various  costs  that  management  should 
recognize  when  it  automates;  and  (3) 
substantial  planning  and  attention  to  detail 
by  management  and  users  can  result  in  a 
much  more  effective  system  for  the 
investment. 

The  types  of  decisions  that  contracts 
professionals  can  and  should  influence 
include: 

•  The  objective  of  automation 
selected  bv  agency  management. 
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Whether  automation  is  intended  to  improve 
efficiency,  to  improve  quality,  to  control 
the  process,  or  to  reduce  jobs. 

•  The  strategy  of  automation  selected 
by  office  management. 

If  automation  is  a  part  of  a  planned 
process  change,  rather  than  being  imposed 
on  top  of  an  existing  system,  it  stands  a 
much  greater  chance  of  successfully 
attaining  objectives.  However,  automation 
in  conjunction  with  a  change  in  process 
requires  much  more  commitment  by 
management,  and  more  involvement  by 
users.  Process  changes  require  significant 
planning,  review  and  evaluation  by 
participants.  It  also  requires  training  and 
technical/functional  support  through  the 
transition  phase  before  and  after  cutover. 

•  Program  and  technical  management 
during  development  and  implemen¬ 
tation. 

Probably  the  single  most-frequent  reason 
why  operational  objectives  are  not  attained 
by  new  procurements  of  most  types  of 
automated  systems  is  lack  of  capable 
technical  people  to  support  the 
procuring  CO.  It  is  very  difficult  for 
government  procuring  offices  to  obtain  and 
retain  sufficient  technical  support.  And  it 
is  difficult  in  the  government  environment 
to  determine  accountability  for  technical 
decisions,  or  for  omissions.  The  CO 
should  ensure  that  he  or  she  has  a  good 
technical  team,  that  they  are  dedicated  to 
and  accountable  to  the  project,  and  that 
they  remain  throughout  the  project. 

•  The  budget  to  change  and  adapt  the 
requirements  and  design  DURING 
development  and  implementation. 


If  the  automation  project  extends  over 
more  than  nine  months  or  a  year,  manage¬ 
ment  should  hold  sufficient  funds  available 
to  adjust  and  modify  the  system,  prior  to 
cutover.  If  the  users  have  altered  their 
requirements,  and  the  specifications  have 
not  been  adjusted,  then  the  government 
must  accept  a  system  that  no  longer  meets 
its  need  -  and  is  not  likely  to  be  successful. 

•  Efforts  to  convert  data  to  the  new 
system. 

In  many  cases,  a  new  system  will  use  data, 
or  provide  data,  that  resembles,  but  is 
slightly  different  from  the  data  in  use,  or 
the  data  needed  by  another  system  or 
process.  It  is  very  important  that  data  be 
analyzed  and  that  the  appropriate  actions 
be  taken  to  either  convert  data  to  a  usable 
form,  or  to  provide  an  appropriate 
translator.  Data  is  one  of  the  end  products 
of  contracts  offices,  and  if  the  data  is 
corrupted  or  requires  manual  intervention, 
then  the  benefits  of  the  investment  are 
diminished. 

•  Data  compatibility  among  systems 
(interoperabilitvV 

Many  contracts  offices  have  elected  to 
purchase  and  use  special-purpose  tools  to 
automate  functions  within  the  office.  Of 
course  the  issue  with  tools  is  that  they 
might  not  be  exactly  appropriate  for  the 
function  of  that  office,  or  that  they  might 
produce  outputs  that  require  manual 
intervention  to  be  used  by  another  process. 
Interface  tools  and  translators  are  not 
difficult  programs  for  vendors  to  write  and 
provide,  but  they  must  be  precise  and 
accurate.  They  can  be  memorable 
headaches. 
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It  is  a  challenge  to  contracts  professionals 
to  take  the  time  to  do  the  analysis  in  the 
detail  sufficient  for  interface  programs  to 
be  written  that  solve  interoperability 
problems. 

The  alternative  to  interoperability  is  for  the 
users  to  have  systems  that  require  them  to 
work  for  the  computer,  rather  than  the 
computer  working  for  the  users. 

•  Training  and  education  ^technical 
and  functionalV 

Training  budgets  are  often  cut,  when  funds 
run  short,  yet  what  good  is  a  new  system  if 
the  users  don’t  understand  and  can’t  use  its 
capabilities?  Unlike  the  experience  with 
shrink-wrapped  software,  users  of 
sophisticated  contracts  software  rarely 
seem  to  be  able  to  grow  into  the  system. 
Instead,  the  system  is  not  used. 

Effective  training  can  be  formal  or 
informal,  but  it  must  include  a  live  system, 
and  an  opportunity  for  users  to  test  the  tool 
or  application  on  a  live  system. 

•  The  budget  for  technical  and 
functional  support  during  transition. 

The  transition  phase  is  the  phase  from 
acceptance  of  the  system,  through  training 
and  cutover,  and  through  the  period  until 
the  users  are  completely  comfortable  with 
all  the  features  of  the  system.  On-site 
vendor  support  during  this  phase  is  very 
important  for  full  utilization  of  the  system 
by  users. 

•  Planning  for  support  staff  after 
transition  fduring  maintenanceV 

Vendors’  prices  for  staff  support  are 


usually  challenged  by  management  because 
they  perceive  the  value  as  being  intangible 
and  low,  and  that  the  prices  are  too  high. 
In  fact,  the  vendor’s  post-transition  support 
is  crucial  to  the  use  of  most  sophisticated 
tools. 

Most  professionals  learn  a  system  by  using 
it,  and  having  a  near-by  expert  to  refer  to 
is  highly  cost-effective,  and  helps  the 
project  go  smoothly. 

•  The  budget  for  changes, 
enhancements,  adaptations  and 
adjustments  after  acceptance 

Most  contracts  offices  are  unique  enough 
that  they  will  want  their  systems  to  be 
customized  and  enhanced.  These  costs  are 
often  small  relative  to  the  others,  yet  they 
can  have  significant  benefits.  Sometimes, 
adaptive  changes  are  necessary  for  the 
system  to  accomplish  its  original 
objectives. 

The  real  lessons  to  be  learned  are  that:  (1) 
the  purchase  price  of  the  software  itself  is 
just  one  component  of  the  various  costs 
that  management  should  recognize  when  it 
automates;  (2)  substantial  planning  and 
attention  to  detail  by  management  and 
users  can  result  in  a  much  more  effective 
system  for  the  investment;  and  (3)  a  little 
investment  in  technical  support  after 
cutover  is  very  worthwhile. 

CONCLUSIONS 

The  number  of  ineffective  or  under-used 
automated  systems  in  support  of  federal 
contracts  offices  is  a  cause  for  concern, 
because  many  of  the  functions  of  contracts 
offices  are  difficult  to  automate.  Although 
contracting  functions  are  complicated  and 
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often  ambiguous,  the  current  automated 
tools  and  systems  that  are  available  can 
provide  improvements  to  office  outputs, 
and  to  the  skill  level  and  professional 
opportunities  of  the  contract  professionals 
in  the  offices. 

If  properly  implemented  and  supported,  the 
functional  software  now  on  the  market  can 
provide  a  major  improvement  to  the 
workload  and  job  satisfaction  of  contracts 
professionals.  However,  if  automation  is 
poorly  planned  and  implemented,  it  can  be 
a  nightmare  to  the  users  and  professionals. 
So  the  ones  with  a  major  stake  in  the  long¬ 
term  success  of  automation  of  contract 
offices  are  the  professionals. 

The  means  to  successfully  procure  and 
implement  automated  tools  and  systems  in 
contracts  offices  is  to  address  the  difficult 
issues  described  above.  This  means  plan¬ 
ning,  budgeting  and  follow-through. 
Because  the  people  with  the  greatest  stake 
in  success  are  the  "users"  now  working  in 
the  offices,  they  should  recognize  the 
issues  involved,  and  actively  ensure  that 
the  issues  are  fully  satisfied  when,  and 
after,  software  is  procured. 
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1 993  ACQUISITION  RESEARCH  SY^^POSIUM 

mXERIEL  ACQUISITION  -  PROCESS  DEFINITION 


ABSTR.\CT 

A  restnictviniing  etfort  at  HQ  .Army  Materiel 
Conunand  (.AMC)  created  the  need  to  clarify 
where  .AMC’s  missions  and  functions  fit 
within  the  acquisition  life  cycle  process. 
Attempt.s  to  analyse  this  process  resulted  in  a 
complex  and  inconsistent  set  of  datsu  We 
needed  a  standardized  set  of  activities  that 
could  be  tied  to  the  life  cycle  process  in  a 
simple,  convenient  way. 

In  1990-91  tlie  Defense  Science  Board 
pro<luced  a  model  of  the  acquisition  process. 
Working  at  HQ  .AMC,  f  rebuilt  the  model, 
retaining  the  D(9D  level  framework, 
expiuiding  it  to  include  all  phases  of  the  life 
eye!,'  acquisition  process  and  revising  it  to 
retlect  the  nev\  series  guidance.  I 

also  included  resource  management  and 
personnel  activities. 

Tlie  solhvare  that  I  used,  .\la.ster  Phumer, 
permitted  me  to  designate  peifonning  ofllces 
juid  define  the  processes  and  tlmciions 
pertbnned  by  the  dii  ectorates  and  QMDs  of 
HQ  AM(.’  in  conte>d  of  the  DOl.)  acquisition 
process,  'fhis  road  map  prov  ides  visibility 
to  s'ubiirocesses  within  the  overall 
acquisition  process  and  shows  the 
inf  .'nelationships  of  offices  and  directorates. 

It  also  can  serve  a.s  a  librar,  ofTD.A  data 
imd  related  infonnation  such  a.s  products, 
customers,  regulations  and  cost  benefit 
sfatement.s. 

Tlie  model  and  softwaie  ciui  be  easily  and 
legallv  u.^ed  throughout  Dt  )D.  The  model  is 

507 


composed  of  dBase  files  and  can  be  iim  on 
PCs.  Tlie  stiiictuiv  of  tlie  model  makes  it 
easy  for  other  commands  or  services  to  adajjt 
the  model  and  build  upon  it. 

INTRODUCTION 

hi  1990  the  Defense  Science  Board  (DSB) 
v^'as  tasked  to  review  the  D(]>D  Materiel 
Acquisition  Process.  Their  objective  was  to 
shorten  the  time  taken  to  acquire  materiel  by 
Originally  the  project  was  to  be 
conducted  in  i  phases,  document  the  existing 
system;  recommend  impi  ovenients  to  shorten 
materiel  acquisition  time;  and  ini, .dement  the 
recommendations.  Tlie  first  pha.se  wa.s  to 
take  3  months  using  persoimel  borrowed  from 
the  seiv  ices. 

Tlie  task  proved  quite  dilficult  and  the  first 
phase  gi  ew  to  consume  about  one  luid  one 
half  years.  Tlie  longer  than  anticipated  time 
caused  a  high  himov  ei  of  personnel  This 
resulted  in  loss  of  institutional  knowledge, 
changes  ir  procedures,  and  disnipted 
commimications.  The  nio.st  devastating 
occurrence  was  the  resignation  of  Secretaiy 
Bettie  and  subsequent  loss  of  political 
support.  Tliis  resulted  in  the  project  b-  ing 
wr.'ipped  up  near  the  end  of  phase  1. 

hi  spite  of  these  problems,  they  did  produce  a 
valid  model  of  the  acquisition  process, 
covering  the  activities  from  identification  of 
niis.<ion  need  to  I(  >1'.  Tlie  model  was  m  the 
form  of  a  pert  chai1  that  shovved  the  tasks 
performed  (840  of  them),  the  logical 
relationships  between  the  tasks,  luid  the  time 


ri-quirefl  to  p-Mlorn)  fho  fJi.sks.  Tlii'  activities 
makinc  up  Uk*  model  are  identified  and 
sequenced  by  tiie  phase  of  the  acquisition 
life  cycle  process.  Ilie  time  data  (time  to 
perfonn  and  man  days  required)  was, 
unfommately,  suspect,  but  the  other  pails  of 
the  model  were  considered  valid  at  the 
completion  of  the  study  Quantities  of  other 
data  were  collected,  but  this  paper  only 
concerns  the  pert  chart. 

At  the  time  this  DSB  etlbil  was  being 
concluded.  Headqiiailers.  .Army  Materiel 
(.  ommand  (flQ  .AMC).  was  undergoing  a 
r  .^orgimi;:ation  and  reduction  'Hie  method 
chosen  to  conduct  this  reorganization  was  to 
have  eiglit  macro-business  process  action 
teams  (PATs).  later  named  quality 
management  boai  ds  (QMBs).  each  map  out 
their  ou  t!  processes  and  look  for  ways  to 
reoi  giuiize  that  would  result  in  manpower 
sa\  ings.  Each  PAT  QMB  proceeded 
accoi  (ling  to  its  own  metliodology  and  used 
Its  owai  foniiafs  and  level-;  of  details  llie 
result  was  eiglit  ditlei  eiil  phuis  that  were 
vei-y  ditlicult.  ifnot  impossible,  to  link 
togerlier  :uid  conipai  e. 

LTG  Tliomas.  die  DCG  for  UDA  at  HQ 
svirs  briefed  on  the  DSB  ettbil  and 
saw  their  acquisition  model  as  a  vehicle  that 
iiiielii  be  used  to  staiidanli.ze  and  define  the 
eidit  dilfering  processes  that  HQ  .AMG  was 
documenting  Die  DSB  model  would  also 
provide  a  complete  picture  of  the  acciuisition 
liK'ces.s  that  could  be  used  to  envelop  HQ 
.AMc  ’s  processes  and  help  define  what  work 
w  e  were  supposed  to  do  and  identity  our 
custonieis 

I  was  tasked  to  import  the  DSB  model  info 
AM<  ■  luid  insert  the  actiMties  described  by 
each  of  the  eiglit  busine.ss  processes  My 
objecfive  was  to  stiuidardi;  e  luid  define  HQ 
AM',  s  processes 


MODEL  DEWLt^PMENT 

Tlie  first  step  was  to  choose  a  sottwai  e 
package  tliat  was  adaptable  to  the  data  and 
could  easily  be  used  on  our  e.xisting 
equipment  and  within  our  limited  budget.  I 
chose  a  sothvai  e  package  that  had  been  used 
for  some  of  the  work  at  the  DSB  and  was 
designed  to  handle  acquisition  related 
intbmiation:  MCSAM  (Marine  Coqjs 
Sy  stems  .Acquisition  Model).  Tliis  softwiu  e. 
now  commercially  ofi'ered  as  Master 
Planner,  was  already  licensed  to  DOD  and 
was  readily  a\  ailalde  from  the  contractor. 
Strategic  Financial  Phuuiing  Sy.sfenis.  Inc. 

Tlie  sothvai  e  w  orked  in  a  DuS  enviroiuiient 
luid  used  dBase  files,  which  had  a  large  user 
suppoit  base.  It  also  had  a  gi  ;iphic.s  and  data 
entry'  and  retrieval  overlay  to  make  data  input 
and  access  very  user  friendly.  Since  the 
Mai  ine  Coips  used  both  .Anny  luid  Navy 
acquisition  systems,  the  sothvai  e  was 
compatible  with  both  acqm.--ition  sy.steins  and 
needed  no  modification  to  use  w  itli  the 
•Anriy's  way  of  acquirinc  inalei  iel  Tlie 
sothvai  e  could  also  be  used  on  any  386 
personal  computer  (a  286  could  be  used  if 
tlie  data  base  were  small). 

Tlie  second  step  w'as  to  revise  the  data 
produced  by  the  DSB  Because  the  model 
took  so  long  to  build  by  the  DSB.  it  was 
made  obsolete  the  day  it  was  releirsed 
becmise  the  acquisition  process  had  changed: 
D.A  published  the  .^000  series  of  acquisition 
policy  So,  with  the  help  of  the  stafi'at  .AMv'. 
I  began  tlie  Utsk  of  updating  the  process  cliail 
[irodiiced  by  tlie  DSB.  Since  a  complete  luul 
thorough  revision  w;is  going  to  be  needed,  I 
also  decided  to  expiuid  the  model  to  include 
Uie  entire  lile  cycle:  from  definition  ofilie 
need  fhrougli  di.spo.sal  of  fh:  equipment.  Tlie 
original  model  ended  :  *  IGG  luid  contained 
8-4(1  activities  Presently  the  frill  updated  life 
cvcie  model  contains  1377  activities 
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Attor  updntins  the  model,  I  tlien  befl;an  to 
ov  erlay  the  acti\  ities  peilbmied  by  tour  of 
the  eiglif  macro-business  processes  at  HQ 
AMl  that  were  directly  related  to  materiel 
acquisition.  I  retmned  the  DOD 
characteristics  of  the  model  and  identified 
those  activities  that  were  eitlier  owned  or 
perfonned  by  HQ  AMC.  I  defined 
ownership  as  that  QMS  that  had  the 
r.'sponsibility  for  integration  of  the  activity’, 
not  necessai  ily  control  or  responsibility  for 
the  activity  [  then  •identified  the  DCS 
otfices  at  HQ  .A_VIC  that  achially  peifomied 
(expended  manpower  on)  the  activity. 

In  .^oine  cases.  HQ  unique  fiinctions  had  to 
be  added.  Most  of  these  HQ  functions  were; 
policy,  integi  ation.  oversight  or  evaluation. 

1  inserted  these  new  activities  in  parallel  to 
the  'ptofliict  generation"  activities  usually 
li  .’ifonned  by  the  progi  ani  managers  (PMs). 
major  .subordinate  conunand.s  (MSC),  or 
contractors  When  1  had  the  data  available,  I 
also  included  peifomianco  by  organizations 
outside  of.AMc'  (e  g  .  TR,-VDOC.  DA).  In 
tins  way,  all  the  HQ  activities  could  be 
defined  in  fenn  of  where  in  the  life  cycle 
model  their  work  was  peiibnned.  who  or 
whai  tr  iggered  the  HQ  activity .  surd  who  or 
what  w.'is  the  ciisteaner  This  would  make  it 
ea.--ier  to  identity  tlie  etlect  or  impoilance  of 
the  Hq  tiinclion  for  piirjjoses  of 
reorgani.’alion.  divestiUire  or  tenmnation  of 
an  acti\  iry. 

.Alter  adding  the  business  processes  tliat 
were  directly  related  to  materiel  acquisition 
to  fh.‘  model.  1  then  attempted  to  add  those 
processes  that  were  less  directly  or 
indirectly  related  to  the  lite  cycle  process. 
Ihe  first  ’indirect"  process  that  I  added  was 
resource  management.  Some  of  these 
actn  ifies  tit  directly  into  the  existing  model 
stnictiire  ;uid  could  be  tagged  directly  to  the 
existing  model  elements  ( >ther  indirect 


activities  had  to  be  added  in  a  parallel 
network  and  only  linked  in  one  or  two 
locatioas  where  specific  activities  did  tie 
into  the  existing  model  stnicture.  When 
handling  resource  management  tasks  that 
were  controlled  by  tlie  timing  of  the  planning, 
programing,  budget,  execution  system 
(PPBES)  I  designated  tliose  tasks  as 
"repetitive"  to  separate  them  from  the  other 
acquisition  related  tasks  that  were  "phase" 
related.  Thus  1  was  able  to  accommodate,  in 
one  cohesive  model,  both  phase  contiolled 
and  calendai’  controlled  tasks.  Using  these 
tecliniques,  I  then  incorporated  some  of  the 
personnel  process  tasks  perfonned  at  HQ 
.AMC. 


DATA  BASE  .AND  MODEL 
DESC’REP^riON 

Tlie  software  is  designed  to  produce  the 
following  types  of  printouts  tuid  reports. 

Netw’ork  diagi  ains  showing  the 
activities  and  their  logical 
relationships. 

G.ANIT  chails  showing  tlie  time 
relationships  of  the  activities. 

.Acti\  ity  repoits  showing  luiy  of  a 
wide  vai  iety  of  possible  infonnation 
fields  tliat  relate  to  each  acti\  ity. 

l.ogic  repoits  showing  the  logical 
connections  between  ta.sks. 

.A  wide  vai  iety  of  resoiuce  repoits 
showing  dollai  s  and  manpower 
requiremeiits  either  associated  to  an 
activity  or  ex^iended  in  the 
peitbnnance  of  a  related  group  of 
activities 
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Cost  reports  that  can  be  depicted  in  a 
tabular  foiinat,  graphical  format,  or  a 
bai'  chart  format.  Several  different 
rt’pes  of  reports  can  be  displayed, 
and  a  program  budget  can  be  ejisily 
accessed  and  manipulated. 

TTie  data  base  can  be  manipulated  (sorted 
and  selected)  mid  printouts  cu.stomi7.ed  just 
as  any  tlBase  printout.  Tlie  most  frequently 
used  printouts  are  already  tbrmatted  for  the 
coin  enience  of  the  user  by  the  Master 
Planner  sothvm  e,  sind  ai  e  easily  configured. 

.Another  feature  of  the  software  is  a  ''library" 
of  tasks  or  actions  which  allows  users  to 
quickly  and  easily  select  specific  paits  of  the 
iaigo  model  and  display  only  specific 
portions  related  to  the  users  needs  and 
orienlation.  It  is  possible  to  '‘tag’’  the 
activities  to  custoinize  the  datn  base  atid 
larae  coinmeiif  fields  m  e  already  provided 
tor  in  the  software.  The  library  of  activities 
has  been  created  througli  extensive  resemxh 
into  Del)  and  other  govenuiieiit  documents, 
specificatioii-s.  mid  ivquii  eriieiits.  It  is  the 
.school  solution  to  nimiy  dilT’rent  acquisition 
scenm  ios,  but  can  also  be  used  to  creat** 
In.-^toricaJ  perspectives  mid  as  a  guide  for 
policy  Direct  acce.ss  to  the  dl3a.se  files 
themselves  also  give  the  user  flexibility  for 
moditv'ina  the  basic  model. 

Ilie  IVU'SAM  model  mid  sotKvm  e  me  D<  )D 
fiiopeity  mid  me  ^“asily  mid  legally 
exportable  to  otlier  D'  >D  offices  mid 
activities,  llie  soflvvm  e  is  designed  to  nm 
m  a  D(  'S  env  ironnienf  mid  will  ftinclion  on 
existinv  286  machme.s  found  iii  most 
aovenunent  offices  However,  if  the  data 
ba.se  becomes  Imme.  the  processing  time  and 
hard  di.-,k  .storage  space  will  become 
practical  linutine  factors  386  machines  m'  * 
leconiniended  Hie  .Master  I’lanner 
solKvaie.  mi  iipai ade  to  Mi'SAM,  will 


provide  extensive  mid  powerful  program 
management  and  organizational  functions 

FITLUE  EFFORTS 

My  next  task  is  to  add  the  remaining 
personnel  related  tasks  mid  then  to  include 
base  operation  mid  information  management 
activities  into  tlie  database.  Concunviitly 
witli  the  addition  of  these  tasks  I  will  need  to 
modify'  the  model  to  match  the  changes 
occuiring  at  HQ  ,AMC.  The  acquisition 
proce.ss  is  dynamic  mid  ever  changing  mid  the 
process  to  update  models  to  reflect  these 
changes  is  mi  ongoing  operation. 

Tlie  model  will  be  used  as  the  baseline  foi 
the  .Acquisition  QMB  at  HQ  .AMC  mid  future 
plmis  include  exiimiding  tlie  data  ba.se  to 
include  details  on  the  MSCs  and  other 
orgmiizations  that  interface  with  the  HQV 

.As  time  mid  resource.s  permit,  the  model 
could  be  expmided  to  be  a  complete 
cenU'alized  libraiy  of  HQ  .AMC 
orgmiizational  iiifoniiation.  The  softwm  e  ha.' 
the  cajialulities  to  include,  mid  integj'ate  with 
tlie  associated  activities,  the  following  types 
of  infonnalion; 

Regulations  mid  laws  that  goveni  or 
coiitiol  the  activities. 

TDA  (table  of  di.'ti  ibiitioii  and 
allovvmices)  data  that  supports  the 
activities 

Mmipowei  that  is  associmed 
(peifonns)  with  the  victiMties. 

Time  nonnally  required  to  peifonii 
the  activity  (.'latistics  could  be 
associated  with  thi>  measure  to  sliow 
the  vm  imice). 
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Products  generated  by  diese 
activities. 

Customers  using  those  products. 

Narrative  stating  the  impact  of 
nonpertbnnance  (utility  of  the 
activiU' );  what  if  not  pertbrmed. 

Adding  and  maintaining  this  data  base 
appeals  to  be  manpower  intensive. 

However,  some  oftliis  data  is  already 
documented  and  periodically  does  get 
updated  in  sepai  afe  data  bases  and  in  hard 
copy.  Creating  and  maintaining  a  centralized 
and  cohesive  data  depository  would  be  a 
new  way  of  doing  business  and  should  save 
effort  in  the  long  tenn.  It  would  be  al')le 
show  the  linkages  of  the  data  and  allow  easy 
access  of  related  files.  To  some  extent,  this 
would  be  a  change  or  reorganization  of 
where  existing  data  is  recorded  and  stored, 
not  a  new  data  gathering  and  storage  effort. 

Tliere  are  some  minor  sottwai  e 
modifications  that  would  make  such  a 
centralized  database  easier  to  create  and 
maintain.  Cue  task  is  to  put  it  on  a  local  area 
network  (LAN)  where  everyone  in  a 
conunand  could  have  easy  access  to  it  for  tlie 
updating  and  retrieval  of  information  that  is 
pertinent  to  them  (the  program  is  L,AN 
ready).  A  program  modification  would  be 
creation  of  a  single  "window"  or  screen  that 
would  let  a  user  have  quick  and  easy  access 
to  all  the  data  fields  that  relate  to  tiie  TDA. 
Several  windows,  for  simplified  data  entry, 
are  already  built  into  the  Master  Planner 
program  (oriented  towards  the  acquisition 
process  rather  than  towards  an 
organizational  or  TDA  structure).  Little 
effort  would  be  required  to  constmct  another 
window  to  bring  together  the  additional 
TDA  elements  described  above. 


BENTIFITS 

'Tlie  model  can  be  very  helpful  in  seiving  as  a 
road  map  to  enal^le  analysis  to  restrticture 
subprocessos  (e.g.,  tlie  finctioning  of  an 
oftlce)  within  the  overall  acquisition  process 
or  laiger  organizational  luiit.  It  also  provides 
the  commander  witli  a  picture  of  his 
command  and  the  inteirelationships  of  his 
ofiices  and  directorates.  Additionally,  it  can 
be  of  assistance  to  acquisition  policy  offices 
to  lay  out  the  stnictuie  of  the  life  cycle 
materiel  acquisition  process  and  those 
players  (jieiformmg  offices)  which  ai  e 
affected  by  existing  policy  and  would  be 
restructured  or  impacted  by  new  policy.  Die 
softwaiv  already  allows  constinction  of 
"what  if  scenai'ios  to  show^  the  stnictui  e, 
time  and  resouixe  requirements  of  alteniate 
environments. 

The  stnictuie  of  the  model  makes  it  easy  for 
other  commands  or  services  to  adapt  tlie  data 
to  describe  the  workings  and  relationships  of 
tlieir  own  offices  and  directorates  as  well  as 
both  subordinate  and  hi.aher  HQ.  Die  model 
has  been  briefed  to  personnel  at  the  Defense 
Systems  Management  College  (DSMC)  and 
die  Air  Force  Systems  Conunand  has  used  if 
as  a  starting  point  for  their  own  model 
building  effort. 

INFORMADCN  Sf^LUCES 

For  information  on  the  commercial  version  of 
tlie  software.  Master  Planner,  you  should 
contact  the  developer; 

Strategic  Financial  Plaiuiing  Systems 

Attn;  Clancy  McQuigg 

2241-R  Tacketts  Mil'l  Dr. 

Woodbridge,  VA  22192-3024 


Phone;  (703)  643-0818 


For  copies  of  the  data  base  and  printouts  of 
the  acquisition  process  chail,  contact; 

Ben  Perchik,  Operations  Research 
Analyst 

HQ  .AMC 

.AMCAQ-AP  (Perchik) 

5001  Eisenhower  Av 
Alexandria,  VA  22333 

(703)-274-8225/DSN-284-8225 

Email: 

bperchik5a'alexandria-enili5. amiy.mil 

SLTVEVL^RY 

Driven  by  the  need  for  a  standardized  and 
cohesive  picture  of  AMC  missions  and 
tasks,  we  have  developed  an  updated  and 
easy  method  to  comprehend  model  of  the 
DOD  acquisition  life  cycle  process.  The 
model,  using  familiar  dBase  files  and 
suitable  for  most  personal  computers,  forms 
the  basis  for  a  library  thsit  can  store  a  variety 
of  information,  all  linked. 

♦  HQ  activities,  both  directly  and 
indirectly  (resource  management  and 
personnel)  related  to  tlie  life  cycle 
process. 

♦  Activities  performed  by  other 
organizations  such  as  the  PMs,  MSCs, 
DA,  Congress,  contractors,  other 
commands,  etc. 

♦  Customers. 

♦  Products. 

♦  Time  to  perform. 

♦  TDA  information. 

♦  Consequences  of  not  performing  the 
task. 

♦  Regulations  governing  task 
performance. 

The  main  uses  of  this  data  are  to  provide: 


♦  A  road  map  of  the  acquisition  process 
to  enable  policy  makers  to  more  easily 
see  tlie  consequences  of  new  policy 
and  changes  to  the  process. 

♦  A  pennanent  data  base  of  TDA 
information  linked  to  the  o\  erall 
missions  and  lunctions  tliat  as.sists  in 
reorganizations  and  dowmsizings. 

♦  A  common  libraiy  of  infonnation  tliat. 
when  put  on  a  local  area  net.  gives 
easy  access  to  offices  to  update  and 
reference. 

The  model  and  software  (Master  Plaimer) 
can  produce  a  wide  variety  of  graphic  and 
tabular  reports  on  selected  mid  soiled  data 
fields.  They  are  easily  and  legally  usable 
and  available  for  modification  and  use 
throughout  DOD. 

AUTHOR 

Ben  Perchik.  Operations  Research  .Analyst 
HQ  AMC,  AMCAQ-/\P 
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STUDY  OF  THE  BELATIONSHIP  BETWEEN  PERSONALITY 
CHARACTERISTICS  AND  ETHICAL  SENSITIVITY  IN  BUSINESS 

Ronald  A.  Ortiz,  Capt,  USAF  David  Patrlllo,  Capt,  USAF 

DPRO,  Martin  Marlatta  ASC  Wrl|ht  Laboratory 


Abstract 

This  research  establishes  a 
statistically  significant 

relationship  between  ethical 
sensitivity,  which  is  the  perception 
that  ethics  and  ethical 

considerations  are  present  in  a 
given  situation,  and  personality 
type,  as  characterized  by  the  Myers- 
Briggs  Type  Indicator  (MBTI). 

A  randomly  stratified  sample 
from  the  membership  of  the  National 
Contract  Management  Association 
(NCMA) ,  resulted  in  a  research 
sample  of  466  respondents.  The 

sample  was  analyzed  by  segmentation 
into  government  (181)  and  private 
industry  (285)  respondents.  The 

average  response  from  the  sample  of 
5.00  on  a  seven-point  scale  for  the 
ethical  sensitivity  questions  is 
considered  high.  Differences  in  the 
perceptions  of  ethical  sensitivity 
were  determined  to  be  statistically 
significant  among  the  government  and 
industry  segments,  and  among  the  ten 
questions  describing  scenarios  faced 
by  contracting  professionals.  In 
the  majority  of  the  questions,  the 
government  segment  reported  higher 
ethical  sensitivities  than  the 
industry  segment.  However,  industry 
reported  higher  ethical 

sensitivities  to  one  question 
involving  an  arithmetic  error 

causing  a  loss  to  the  contractor. 
There  was  a  statistically 
significant  difference  in  the  MBTI 
distribution  of  the  sample  compared 
to  the  data  bank  of  the  Center  for 
the  Application  of  Psychological 
Type  (CAPT) .  The  research  sample 
contained  more  introverted,  sensing, 
and  thinking  types  than  the  CAPT 
sample.  Among  the  government 


segment,  it  was  established  that 
those  favoring  intuition,  and 
intuition  and  thinking  as  their 
cognitive  functions,  exhibit  higher 
degrees  of  ethical  sensitivity  than 
other  personality  types. 

Recommendations  for  additional 
research  are  provided. 

Introduction 

During  the  1980s,  the  media 
exposed  the  public  to  several 
scandals  involving  the  defense 
industry.  In  1991,  UNISYS 
Corporation  was  found  guilty  of 
civil  and  criminal  charges  of 
conspiracy  to  defraud  the 
government,  bribery,  and  filing 
false  claims  on  government 
contracts.  These  convictions 
resulted  in  $190  million  in  fines, 
penalties,  and  relinquished  profits. 
Meanwhile,  the  Ill-Wind 
investigations  which  helped  to 
convict  UNISYS  also  resulted  in 
additional  convictions  of  51 
persons,  and  six  other  companies  for 
various  forms  of  malfeasance  on 
other  defense  contracts  (14:319). 
In  1990,  Under  Secretary  of  Defense 
for  Acquisition  John  Betti  resigned 
under  fire  after  it  was  discovered 
that  DoD  officials  misrepresented 
the  extent  and  magnitude  of  the 
financial  problems  associated  with 
the  Navy’s  A-12  program  (9:44).  The 
program  was  subsequently  canceled. 

There  have  been  similar 
occurrences  in  the  Savings  and  Loan 
(S&L)  Industry  and  among  Wall 
Street's  top  brokerage  houses.  It 
is  estimated  that  the  Federal 
Deposit  Insurance  Company’s  bank 
fund  has  paid  out  $56  billion  to  the 
depositors  of  failed  S&Ls  and  that 
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an  infusion  of  an  additional  <70 
billion  will  be  needed  (4:30-32). 
Michael  Milken,  a  noted  Wall  Street 
stockbroker  from  the  Drexel  Burnham 
Company,  pleaded  guilty  to  six 
felony  counts  involving  insider 
trading,  paid  fines  and  restitution 
totaling  $600  million,  and  was 
sentenced  to  ten  years  in  prison 
(15:30) . 

As  a  result,  the  ethical 
behavior  of  individuals  and 
organizations  has  come  under 
increased  public  scrutiny.  In  fact, 
according  to  a  1987  Time  Magazine 
study,  76%  of  the  American  public 
saw  a  lack  of  business  ethics  in 
managers  as  contributing  to  the 
decline  of  U.S.  moral  standards 
(7:261). 

Significance  to  Qovarnmant 
Acquisitions 

In  the  midst  of  this  turmoil, 
the  federal  government  increased  its 
regulatory  interest  in  ethics, 
especially  with  regard  to 
government,  and  particularly 
Department  of  Defense  (DoD) 
acquisitions.  Public  Law  96-903, 
unanimously  passed  by  Congress  and 
signed  into  law  in  1960, 
established  a  code  of  ethics  for  all 
government  employees.  The  code 
contains  ten  basic  principles 
including  upholding  the  Constitution 
of  the  United  States,  exposing 
corruption  whenever  possible,  and 
putting  loyalty  to  the  highest  moral 
principles  and  to  country  above  all 
else.  In  addition.  Part  3  of  the 
Federal  Acquisition  Begulation  (FAR) 
contains  further  clarification  and 
implementation  procedures  related  to 
receiving  or  soliciting  gratuities, 
disclosing  proprietary  information, 
post-employment  restrictions  of 
certain  former  and  retired  DoD 
employees,  and  other  standards  of 
conduct  (11:16,  307-16,341). 

The  DoD  became  further 
involved  in  the  area  of  regulating 
ethics  when  it  issued  Directive 


Individuals 
violations 
subject  to 


5500.7,  Standards  of  Conduct,  in  May 
1987.  The  directive  prohibits  using 
inside  information  for  personal 
gain,  prohibits  conflicts  of 
interest,  requires  submissions  of 
financial  interest  and  affiliation 
statements,  prohibits  the  release  of 
acquisition  information,  restricts 
outside  employment  opportunities  for 
DOD  personnel,  and  establishes 

agency  ethics  officials  and 
committees  (3:1-24).  In  an  attempt 
to  provide  further  guidance  to 
government  procurement  agencies  and 
the  defense  industry.  Congress 

passed  the  Procurement  Integrity  Act 
in  1969,  which  imposed  both  civil 
and  criminal  penalties  for 
disclosing  competition  sensitive 
information  to  unauthorized  sources. 

found  guilty  of 
under  this  act  are 
fines  of  up  to  <100,000 
and  a  maximum  prison  sentence 
five  years  (13:23) . 

With  the  passage 
the  implementation  of 
companies  have  recognized  the 

importance  of  integrating  ethical 
decision  making  into  the  corporate 
culture.  According  to  a  survey 

cited  by  Harrington,  63%  of  the 
Fortune  500  Chief  Executive  Officers 
believe  that  a  strong  ethical 

corporate  culture  is  directly 

related  to  developing  a  strategic 

advantage  that  can  result  in 
long-term  benefits  and  profitability 
(5:21).  Companies  are  concerned  with 
unethical  behavior  because  it  can 
lead  to  adverse  public  opinion, 

governmental  intervention  in  the 
form  of  oversight  and  regulation, 
adverse  organizational  costs  in  the 
form  of  lost  profits  and  goodwill, 
monetary  penalties,  criminal 

penalties,  and  even  the  loss  of 

contracts . 

The  Packard  Commission’s 

February  1986  Interim  Report 

recognized  that  there  was  public 
concern  over  'procurement 

irregularities'  and  suggested  that 
'effective  self-governance  might 


of 


of  laws  and 
regulations. 
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help  to  curb  industry  misconduct" 
(1:1).  In  response  to  the 
Commission’s  preliminary 
recommendations,  representatives 
from  eighteen  defense  companies 
drafted  six  principles  which  are  now 
known  as  the  Defense  Industry 
Initiative  on  Business  Ethics  and 
Conduct  (DID.  The  signatory 
companies  pledged  to  promote 
programs  and  policies  associated 
with  a  code  of  ethics,  ethics 
training,  internal  reporting  of 
misconduct,  self-governance, 
industry  responsibility,  and  public 
accountability  (2:1).  As  of 
February  1991,  fifty-five  defense 
companies  have  become  participants 
in  the  DII  (Defense  Industry 
Initiative,  1991:1). 

Companies  are  not  alone  in 
recognizing  the  importance  of 
promoting  ethical  awareness. 
Professional  associations  have  also 
adopted  codes  for  ethical  behavior 
as  part  of  their  by-laws.  For 
example,  the  Hational  Contract 
Management  Association  (NCMA)  has 
established  six  ethical  standards 
their  members  are  obligated  to 
uphold.  The  NCMA  Code  of  Ethics 
promotes  behavior  and 
professionalism  among  its 
membership . 

Ethical  Senaitlvlty 

Establishing,  promoting,  and 
encouraging  ethical  behavior  in  an 
organization  is  a  complicated 
process  which  begins  with  an 
assessment  of  the  ethical  awareness 
of  employees  prior  to  implementation 
of  a  formal  program.  This  ethical 
baseline  is  necessary  for  management 
to  determine  which  areas  of  ethical 
consideration  warrant  their  concern, 
and  to  develop  appropriate  methods 
to  influence  employee  attitudes  and 
behavior  concerning  ethical 
practices . 

Ethical  baselining  requires 
management  to  assess  its  employees 
perceptions  of  the  degree  to  which 


ethics  and  ethical  considerations 
are  a  part  of  their  daily 
work-related  activities.  This  leads 
to  a  new  ethical  construct  which  is 
not  addressed  in  ethics  literature. 
The  degree  to  which  one  perceives 
one's  decisions  and  actions  as  being 
affected  by  necessary  ethical 
judgments  will  be  referred  to  as  the 
individual’s  ethical  sensitivity. 
Take,  for  example,  a  situation  where 
a  contract  administrator  (CA)  needs 
the  signature  of  an  administrative 
contracting  officer  (AGO)  on  a 
particular  document.  With  the  AGO 
unavailable  and  immediate  completion 
of  the  document  necessary,  the  GA  is 
faced  with  the  option  of  signing  the 
AGO’s  name  without  his/her  knowledge 
or  permission.  Ethical  sensitivity 
does  not  address  whether  or  not  it 
is  ethical  to  sign  the  AGO's  name. 
Ethical  sensitivity  does  address 
whether  the  GA  perceives  that  the 
decision  of  whether  or  not  to  sign 
the  name  of  the  AGO  is  even  a 
question  of  ethics.  The  GA  may  view 
this  decision  in  terms  of  the 
oropriety  (right  and  wrong)  or 
morality  (good  and  bad)  of  signing 
the  name,  or  may  not  even  consider 
those  issues  and  view  it  purely  as  a 
business  decision  involving  the 
practicality  of  getting  the  document 
processed  and  assuming  the 
responsibility.  The  degree  to  which 
the  GA  believes  that  ethics  are 
involved  in  the  decision,  regardless 
of  what  that  decision  is,  is  the 
degree  of  the  GA’s  ethical 
sensitivity. 

The  range  of  ethical 
sensitivities  among  the  employees  of 
an  organization  establishes  the 
baseline  which  management  must 
consider  in  developing  and 
implementing  ethical  training 
programs . 

Sptclfic  Problem  Area 

Management,  concerned  about 
their  employees’  sensitivities  to 
ethical  issues  involving  job  actions 
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and  decisions,  should  consider  two 
questions  in  evaluating  the 
organization's  ethical  baseline. 
First,  what  accounts  for  the 
differences  in  ethical  sensitivity 
among  employees?  Second,  is  there  an 
indicator  which  will  accurately 
predict  an  individual’s  ethical 
sensitivity?  The  assumed  notion  is 
that  management  will  be  more 
effective  if  it  understands  the 
factors  which  bear  upon  an 
individual's  ethical  sensitivity  and 
tailors  programs  to  respond  to  those 
factors . 

For  example,  an  individual 
whose  ethical  sensitivity  is 
strongly  linked  to  a  religious 
upbringing  may  respond  favorably  to 
an  ethical  awareness  program  which 
heavily  stresses  the  moral 
implications  of  ethical  behavior. 
With  another  individual,  however, 
ethical  sensitivity  may  be  linked 
solely  to  the  practical 
considerations  of  risk  and  risk 
avoidance.  In  this  case,  the 
program  might  be  aimed  at  the 
negative  repercussions  of  the 
discovery  of  unethical  behavior. 

Clearly  the  aim  and  emphasis 
of  these  two  examples  would  be 
different.  The  discovery  of  some 
predictable  indicator  of  an 
individual’s  ethical  sensitivity 
would  aid  management  in  targeting 
groups  of  employees  with  like 
characteristics  related  to  ethical 
sensitivity  and  structure  programs 
which  would  effectively  influence 
those  groups. 

Problem  Statement 

This  research  explores  the 
notion  that  there  are  indicators 
which  can  help  management  predict 
the  ethical  sensitivity  of  an 
individual  and  understand  the 
factors  from  which  the  sensitivity 
is  formed.  The  indicator  under 
consideration  is  personality  type  as 
characterized  by  the  Myers-Briggs 
Type  Indicator  (MBTI)  (11).  This 


leads  to  the  following  problem 
statement,  the  answer  to  which  is 
the  objective  of  this  research: 

Is  an  Individual’s  personality  type, 
as  characterized  by  the  MBTI,  an 
accurate  predictor  of  that 
individual’s  ethical  sensitivity? 

Among  other  measures,  the  MBTI  model 
of  personality  type  characterizes 
the  way  individuals  perceive  and 
judge  the  world.  It  recognizes  that 
there  are  distinct  processes  for  the 
functions  of  perception  and  judgment 
and  observes  that  individuals  will 
exhibit  a  preference  toward  one 
process  over  another  when  given  a 
choice.  Behavior  patterns  in 
individuals  are  affected  by  these 
functional  preferences  and  the 
combination  of  preferences  manifest 
themselves  in  behavioral  patterns 
which  can  be  characterized  into 
distinct  personality  types. 

The  MBTI  assumes  that  the 
preferences  are  influenced  by  both 
genetic  and  social  factors.  As 
such,  the  model  is  less  concerned 
about  the  origin  of  these 
preferences,  addressing  itself  more 
to  the  manifestation  of  these 
preferences  in  behavioral  patterns 
which  can  then  be  recognized  and 
understood.  Application  of  the 
model  involves  the  study  of  possible 
relationships  between  the  functional 
preferences  and  combinations  of 
preferences,  as  manifested  by 
behavioral  and  attitudinal  patterns, 
and  various  dependent  variables, 
such  as  ethical  sensitivity. 

Reiearch  Question 

This  research  investigates  the 
possible  existence  of  relationships 
between  personality  type  and  ethical 
sensitivity  among  contract 
professionals.  The  research 
question  is  ated  as  follows: 

What  la  tha  ralatlonship  batwtan 
athioal  aansltivlty  and  personality 
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typt ,  af  ffleaaurtd  by  th« 
Ifyarfl-Brigga  Type  Indicator,  for 
contract  profaaalonala? 

Personality  type  is  chosen  as  the 
independent  variable  and  ethical 
sensitivity  serves  as  the  dependent 
variable.  The  research  teats  for 
statistically  significant 
relationships  between  these 
variables  among  a  sampling  of 
contract  professionals  from  the 
membership  of  the  National  Contract 
Management  Association. 

Subsidiary  Quaitiona 

The  first  task  of  this 
research  is  to  determine  the  degree 
to  which  contracting  professionals 
apply  ethical  considerations  to  the 
judgment  of  their  actions  and 
decisions.  The  results  of  this 
analysis  determines  an  individual’s 
ethical  sensitivity.  Then,  the 
sample  is  categorized  by  personality 
type  and  the  components  of 
personality  type,  as  described  by 
Myers-Briggs  type  theory,  and 
related  to  ethical  sensitivity  to 
determine  if  statistically 
significant  relationships  are 
present.  This  analysis  is  guided  by 
a  series  of  subsidiary  questions, 
some  of  which  are  pivotal  to  the 
answer  to  the  fundamental  research 
question.  Pivotal  subsidiary 
questions  are  those  whose  answers 
directly  affect  the  answer  to  the 
research  question. 

The  following  are  the 
subsidiary  research  questions  for 
this  analysis.  Note  that  each 
question  except  the  first  can  be 
formed  into  a  hypothesis  statement 
which  can  be  tested  by  statistical 
analysis.  Since  the  model  of 
ethical  sensitivity  is  a  new  one. 
the  answer  to  the  first  question 
provides  only  descriptive 
information.  There  is  no  data  about 
ethical  sensitivity  in  the  general 
population  with  which  to  compare  it. 


1.  What  are  the  characteristics  of 
ethical  sensitivity  among  survey 
respondents? 

2.  Is  there  a  statistically 
significant  difference  in  the 
distribution  of  personality  type,  as 
characterized  by  the  Myers-Briggs 
Type  Indicator,  among  survey 
respondents  and  the  general 
population? 

3.  Is  there  a  statistically 
significant  relationship  between 
ethical  sensitivity  and  the 
individual  components  of  personality 
type,  as  characterized  by  the 
Myers-Briggs  Type  Indicator,  among 
survey  respondents  (pivotal)’ 

4.  Is  there  a  statistically 
significant  relationship  between 
ethical  sensitivity  and  specific 
combinations  of  personality 
components,  as  characterized  by  the 
Myers-Briggs  Type  Indicator,  among 
survey  respondents  (pivotal)? 

5.  Is  there  a  statistically 
significant  relationship  between 
ethical  sensitivity  and  dominant 
function  preferences,  as 
characterized  by  the  Myers-Briggs 
Type  Indicator,  among  survey 
respondents  (pivotal)? 

6.  Is  there  a  statistically 
significant  relationship  between 
ethical  sensitivity  and  the  sixteen 
personality  types,  as  characterized 
by  the  Myers-Briggs  Type  Indicator, 
among  survey  respondents  (pivotal)? 

Methodology 

This  research  was  conducted  by 
analysis  of  responses  to  a  survey 
addressing  ethical  sensitivity  and 
psychological  type.  The  sampling 
frame  was  the  membership  of  the 
National  Contract  Management 
Association,  which  in  part  sponsored 
the  research  and  provided  a  mailing 
list  of  its  membership.  Of  the 
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23,900  names  available,  1550  were 
randomly  selected  to  receive  the 
survey.  There  were  610  responses 
(39%) ,  466  of  which  were  usable  for 

this  analysis  (30%) . 

Survey  Inetrumenta .  In 

examining  the  relationship  between 
personality  type  and  ethical 
sensitivity,  personality  type  is 
determined  to  be  the  independent 
variable.  This  variable  was 

measured  using  the  Myers-Briggs  Type 
Indicator.  The  dependent  variable 
is  ethical  sensitivity  which  is  a 
new  construct  developed  as  part  of 
this  research.  Measurement  of  this 
variable  required  formulation  of  a 
new  survey  instrument  which  could 
accurately  gauge  the  degree  to  which 
an  individual  perceives  that  the 
determination  of  actions  or 
decisions  in  a  particular  situation 
requires  some  ethical  consideration. 

A  review  of  information  from 
the  Center  for  Business  Ethics  at 
Bentley  College  and  several  defense 
contractor  ethical  handbooks  were 
used  in  developing  ten  scenarios 
which  represented  likely  situations 
that  contracting  professionals  might 
encounter  in  the  course  of 
performing  their  jobs.  This  survey 
measures  ethical  sensitivity  through 
the  respondent’s  answer  to  the 
following  question  relating  to  each 
of  ten  separate  scenarios:  'To  what 
extent  do  you  agree  or  disagree  that 
ethical  considerations  are  involved 
in  making  the  following  decision’* 
The  answers  are  arranged  on  a 
seven-point  Likert  as  follows:  I) 

strongly  disagree:  2)  disagree;  3) 
slightly  disagree:  4)  neither  agree 
or  disagree;  5)  slightly  agree;  6) 
agree;  7)  strongly  agree.  The 

ethical  sensitivity  variable,  then, 
is  continuous,  with  potential  mean 
scores  ranging  from  1  to  7.  The  mean 
score  of  the  answers  to  the  ten 


scenarios  is  the  respondent’s 
ethical  sensitivity  score. 

Survey  validity  was  addressed 
the  instrument  trial  test.  This 

trial  test  was  administered  to 
nineteen  Air  Force  Institute  of 
Technology  graduate  students  in  the 
Contract  Management  program  of 
study.  Their  responses  suggested 
the  instrument  was  indeed  measuring 
ethical  sensitivity  in  the 
respondents.  This  was  confirmed  in 
a  subsequent  session  where  the 
respondents  discussed  their 
perceptions  of  what  they  were  being 
asked  to  provide  in  response  to  the 
scenarios.  Their  feedback  provided 
the  assurance  that  they  gauged  the 
degree  to  which  ethical 
considerations  were  necessary  in 
making  a  decision  with  regard  to  the 
scenario . 

Analysis  of  tha  Data. 

Statistical  analysis  of 
patterns  of  ethical  sensitivity  with 
regard  to  different  independent 
variables  required  the  use  of  mean 
analysis  of  different  populations 
(McClave  and  Benson,  1991:393-453). 
In  all  cases,  a  statistical 
confidence  level  of  95%  +/-  5%  was 
sought . 

Damographlc  Analysis.  The 
demographics  were  grouped  into  two 
major  areas:  personal 
characteristics  and  job 
characteristics.  The  personal 
characteristics  consist  of  gender, 
age,  education  level,  and  ethnic 
origin.  The  job  characteristics  are 
comprised  of  job  title,  position 
level,  and  years  of  experience. 

Tables  1  and  2  summarize  the 
demographic  makeup  of  the 
respondents . 
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TABLE  1 

PERSONAL  CHARACTERISTICS  OF  RESPONDENTS 
(N>466:  181  QOV;  285  INO) 


Government  Private  Industry 

Gender 


Male 

48 . 62% 

( 

88) 

72.63% 

( 

207) 

Female 

51.38% 

( 

93) 

27 . 37% 

( 

78) 

Age 

less  25  Years 

1 . 66% 

( 

3) 

1 . 40% 

( 

4) 

26  -  35 

19.34% 

( 

35) 

17.89% 

( 

51) 

36  -  45 

44.75% 

( 

81) 

31.93% 

( 

91) 

46  -  56 

26.52% 

( 

48) 

27.72% 

( 

79) 

over  55  Years 

7.73% 

( 

14) 

21.06% 

( 

60) 

Educational  Level 

High  School 

1 1 . 60% 

( 

21) 

5.96% 

( 

17) 

Associate 

11.05% 

20) 

7.02% 

( 

20) 

Bachelor 

28.73% 

( 

52) 

40.70% 

( 

116) 

Masters 

46.41% 

{ 

84) 

43.16% 

( 

123) 

Doctoral 

2.21% 

( 

4) 

3.16% 

( 

9) 

Ethnic  Origin 

Caucasian 

87.85% 

( 

159) 

94.74% 

{ 

270) 

Black 

5 . 52% 

( 

10) 

2.81% 

( 

8) 

Hispanic 

2.21% 

( 

4) 

1.40% 

( 

4) 

Oriental 

2.21% 

( 

4) 

1 . 05% 

( 

3) 

Other 

2.21% 

( 

4) 

0 . 00% 

( 

0) 

TABLE  2 

JOB  CHARACTERISTICS  OF  RESPONDENTS 
(N=4e6:  181  QOV;  285  IND) 


Government  Private  Industry 

Job  Title 


Admin/Contracting  Officer 

65. 

19% 

( 

118) 

69. 

12% 

( 

197) 

Buyer/Purchasing  Agent 

14. 

37% 

( 

26) 

17. 

89% 

( 

51) 

Clerical 

1. 

66% 

( 

3) 

0. 

35% 

( 

14) 

Cost/Price/Financial  Analyst 

5, 

,52% 

( 

10) 

9. 

12% 

( 

1) 

Other 

13. 

26% 

{ 

24) 

18. 

25% 

( 

47) 

Position  Level 

Non  Supervisory 

56, 

,91% 

( 

103) 

31, 

,22% 

( 

89) 

Manager /Supervisor 

31 , 

,49% 

( 

57) 

50, 

.53% 

( 

144) 

Executive 

11. 

,60% 

( 

21) 

18. 

.25% 

( 

52) 

Years  of  Experience 

0  -  5  years 

16, 

,02% 

( 

29) 

16 

.  14% 

( 

46) 

6  -  10 

27 

,07% 

( 

49) 

18. 

.95% 

( 

54) 

11  -  15 

30, 

.39% 

( 

55) 

25 

.96% 

( 

74) 

16  -  20 

19 

.89% 

( 

36) 

23 

.86% 

( 

68) 

Over  25 

6 

.63% 

( 

12) 

15 

.09% 

( 

43) 
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Ethics  Policy  Questions •  The 
objective  of  the  research  in  this 
area  is  to  determine  the  extent  that 
companies  have  written  policies  on 
ethics  and  to  obtain  information  on 
the  individual’s  perceptions 
regarding  different  aspects  related 
to  those  policies.  With  respect  to 
ethical  policies,  89%  (161  of  181) 
of  government  respondents  indicate 
their  organizations  have  written 
policies.  It  should  be  noted  that 
18  of  the  20  government  respondents 
who  indicate  their  organization  do 
not  have  a  written  policy  governing 
ethical  behavior  are  from  the 
federal  government  level.  This  is 
unusual  since  ethics  policies,  and 
programs  which  publicize  and  require 
training  of  those  policies,  have 
been  mandated  by  statute  at  the 
federal  government  level.  These 
respondents  seem  unaware  of  those 
policies.  As  for  industry,  85%  (242 
of  285)  indicate  their  organizations 
have  written  ethical  policies.  Of 
the  285  industry  respondents,  20% 
(58)  are  considered  small 
businesses.  Only  about  half  of  the 
small  business  respondents  indicate 
their  organizations  have  written 
policies. 

Analysis  of  Ethical 
Sansitivity  and  Parsonality  Typt 

Ethical  Sansitivity.  The 
first  subsidiary  question  deals  with 
the  characteristics  of  the  ethical 
sensitivity  scores  among  the  survey 
respondents.  The  mean  ethical 
sensitivity  score  is  derived  for 
each  scenario  from  all  466 
responses.  These  scores  indicate 
that  the  respondents  as  a  group 
judged  some  scenarios  to  require  a 
greater  degree  of  ethical 
consideration  than  others.  A 
comparison  of  the  mean  score  of  each 
scenario  with  the  mean  scores  of 
each  of  the  other  nine  scenarios 
reveals  a  statistically  significant 
difference  in  the  ethical 
consideration  required  of  the  two 


compared  scenarios  in  82.2%  of  the 
cases  (37  of  the  45  comparisons) . 
Government  employees  in  the  sample 
report  higher  ethically  sensitivity 
responses,  overall.  The  literature 
substantiates  that  ethical  awareness 
and  training  originated  and  were 
driven  by  the  public  sector.  It  is 
reasonable  to  believe  that  their 
longer  emphasis  on  the  subject  and 
the  maturity  of  their  programs  would 
combine  to  make  them  more  sensitive 
to  the  ethical  considerations 
involved  in  given  situations. 

There  is  only  one  scenario  in 
which  the  private  industry 
respondents  had  a  higher  level  of 
ethical  sensitivity  than  government 
respondents.  In  the  scenario,  a 
government  contracting  officer, 
while  reviewing  a  final  negotiation 
summary  document  provided  by  the 
contractor,  discovers  an  error  in 
favor  of  the  government.  The 
question  concerns  the  ethical 
consideration  involved  in  the 
contracting  officer’s  decision  of 
whether  or  not  to  notify  the 
contractor  of  the  error.  Here, 
private  industry  respondents  were 
more  likely  to  view  the  decision  as 
a  question  of  ethics  than  were 
government  respondents.  This  is 
consistent  with  Jones’  notion  of 
moral  proximity,  which  states  that 
one  is  likely  have  a  higher  level  of 
morally  intensity  over  issues  that 
affect  them  (6:371).  In  this 
scenario,  the  contractor  can  be  the 
victim  or  beneficiary  of  the 
government’s  action  and,  therefore, 
is  sensitive  to  the  ethics  involved 
in  that  decision. 

The  discovery  of  a  significant 
difference  between  the  ethical 
sensitivities  of  government  and 
private  industry  will  be  used  in  the 
analysis  of  the  relationship  between 
ethical  sensitivity  and  personality 
type.  The  further  stratification  of 
the  data  by  government  and  industry 
may  uncover  statistically 
significant  relationships  that  would 
otherwise  be  masked  when  the 
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relationships  are  examined  without 
government  and  industry  separation. 

MBTI  Type  Distribution.  Next, 
the  research  seeks  to  determine  if 
there  is  a  difference  between  the 
distribution  of  personality  types  in 
the  sample  and  the  general 
population.  A  1985  article  in  The 
Journal  of  Psychological  Type  offers 
three  different  data  banks  of  MBTI 
respondents  which  are  drawn  from 
some  cross-section  of  the  general  US 
population,  (8:3-9).  Each  data 
bank,  however,  contains  its  own 
particular  bias.  Of  the  three,  the 
data  bank  selected  for  comparison  is 
maintained  at  the  Center  for  the 
Application  of  Psychological  Type 
(CAPT) .  This  bank  has  over  23,000 
records  taken  using  Myers’  Form  F 
type  indicator,  and  almost  16,000 
others  using  the  Form  G  type 
indicator.  The  respondents  in  these 
groups  were  significantly  weighted 
toward  persons  with  some  amount  of 
higher  education.  This  bias  tends 
to  create  a  high  percentage  of 
introverts  and  intuitives  who  are 
more  likely  to  go  on  to  college  and 
beyond . 

A  Chi -squared  comparison  of 
the  populations  reveals  that  there 
are  statistically  significant 
differences  in  the  MBTI  type 
distributions  of  the  two 
populations.  The  research  sample  is 
shown  to  contain  more 
sensing-thinking  (ST)  types  than  the 
CAPT  data  bank,  particularly  more 
introverted  sensing-thinking  types 
(1ST)  among  women. 

Relationship  of  Ethical 
Sensitivity  and  MBTI  Personality 
Types.  The  next  four  subsidiary 
questions  deal  specifically  with  the 
relationships  between  components  of 
MBTI  personality  type  and  ethical 
sensitivity.  The  data  revealed  that 
there  are  significant  relationships 
between  components  of  personality 
type  and  the  respondents’ 
perceptions  of  their  ethical 


sensitivity.  Specifically, 
comparison  of  the  means  revealed 
that  intuitive  types  (Ns) , 
particularly  among  government 
respondents,  show  significantly  more 
ethical  sensitivity  than  any  of  the 
ether  types.  Further, 
intuitive-thinkers  (NTs)  tend  to  be 
more  ethically  sensitive  than  any 
other  cognitive  sets  of  personality 
preferences.  This  relationships  are 
statistically  proven  to  a  confidence 
level  of  95%. 

There  appears  to  be  a 
relationship  between  ethical 
sensitivity  and  thinking  types  (Ts) , 
as  well,  but  this  relationship 
cannot  be  statistically  supported. 
Comparison  of  the  means  reveals 
confidence  levels  just  under  the 
required  95%.  Also,  there  is  a 
contradiction  along  the 
thinking-feeling  scale  among 
government  and  industry  respondents. 
Government  thinkers  seem  to  be  more 
ethically  sensitive  than  government 
feeling  types.  Among  industry 
respondents,  however,  feelers  showed 
higher  ethical  sensitivity  responses 
than  thinkers.  This  apparent 
contradiction  necessitates  further 
research  into  the  relationship 
between  ethical  sensitivity  and  the 
thinking-feeling  preference  scale. 

There  were  no  other 
statistically  significant 
relationships  revealed  in  the 
research.  In  fact,  analysis  of  the 
sixteen  personality  types  and  their 
relationship  to  ethical  sensitivity 
could  not  be  performed  due  to  the 
low  frequency  of  responses  among  a 
number  of  the  types  (less  than  three 
in  many  cases) .  Analysis  with  these 
low  frequencies  was  determined  to  be 
inconclusive. 

Conclusions 

The  research  question  is 
answered  in  the  following  way: 
There  is  a  statistically  significant 
relationship  between  ethical 
sensitivity  and  personality  type,  as 
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characterized  by  the  Myers-Briggs 
Type  Indicator  (MBTI) .  This 
relationship  is  established  by 
showing  significant  correlations 
between  intuitives  (Ns)  and 
intuitive-thinking  types  (NTs) ,  and 
their  ethical  sensitivity  responses 
on  the  ten  scenarios. 

The  correlation  between 
intuition  and  ethical  sensitivity 
reported  by  this  data  seems  sensible 
with  respect  to  the  characteristics 
of  intuitive  types.  Intuitives  are 
prone  to  look  beyond  the  objective 
description  of  a  situation  toward 
the  meanings  of  what  they  perceive. 
They  seek  ‘the  broadest  view  of  what 
is  possible  and  insightful'  given  a 
particular  situation  (10:13).  While 
reading  the  ten  scenarios  posed  in 
the  survey,  intuitives  are 
instinctively  assigning  meaning  to 
the  events  as  they  unfold  in  the 
scenario.  They  consider  the 
possible  consequences  of  varying 
decisions  and  actions,  and  their 
insight  creates  a  notion  about  the 
ethical  considerations  that  should 
be  involved  in  formulating  those 
decisions  and  actions.  Their 
perceptions  about  what  requires 
ethical  consideration  are  formed 
from  the  notions  they  get  when 
reading  the  scenario,  not  an 
objective  comparison  of  the 
scenario’s  events  ana  what  they  know 
of  ethics  policy.  Their 
preoccupation  with  possibilities 
makes  them  apt  to  see  the  potential 
ethical  consequences  in  most 
situations  which,  therefore,  iray 
create  a  higher  degree  of  ethical 
sensitivity. 

Recommendations 

Enhancement  of  the  Survey 
Instrument.  Ethical  sensitivity 
scores  were  based  on  the  responses 
of  the  survey  population  regarding 
scenarios  that  contract 
professionals  may  encounter  in  the 
performance  or  their  duties.  The 
scenarios  involved  both  government 


and  industry  employees  who  were 
facing  decisions  that  may  or  may  not 
require  ethical  considerations. 
Future  researchers  should  consider 
making  the  scenarios  as  generic  as 
possible  in  order  to  eliminate  any 
potential  bias  of  business  settings 
relating  to  the  ethical  sensitivity 
scores.  The  scenarios  should  be 
written  such  that  there  is  no 
distinction  made  between  government 
or  industry  employees.  Another 
recommendation  is  to  expand  the 
scope  of  the  survey  by  including 
scenarios  about  different  business 
practices  involving  bribery, 
fairness,  honesty,  confidentiality, 
advertising,  coercion,  or 

self-interest  .  The  scenarios  would 
no  necessarily  relate  to  the 
defense  acquisition  industry  and 
could  be  adopted  from  other  related 
research  efforts. 


Ethical  Policiat.  The 
research  established  that  a  large 
majority  of  the  organizations  hav’e 
written  ethical  policies  to  guide 
ethical  behavior.  Also,  the 
policies  are  distributed  to  the 


employees,  training  is  conducted 
regarding  the  policies,  and  a  large 
majority  of  employees  agree  or 
strongly  agree  with  the  policies. 
Further  research  should  examine  the 
differences  between  educational  and 
training  programs  that  are  conducted 
by  various  organizations.  The  data 
should  then  be  combined  with  ethical 
sensitivity  and  MBTI  personality 
type  data  to  examine  whether 
education  and  training  are 
moderating  variables  which  affect 
ethical  sensitivity.  Furthermore, 
an  analysis  of  the  data  should  be 
performed  to  determine  whether  these 
programs  influence  some  personality 
types  more  than  others,  and  visa 
versa . 

4^ddition«l  MBTI  Reatarch. 
Subsidiary  Question  3  examined 
whether  a  statistically  significant 
relationship  between  ethical 
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sensitivity  and  individual 
components  of  personality  types 
exists.  As  previously  discussed  in 
Chapters  IV  and  V,  the  mean  ethical 
sensitivity  score  for  government 
thinkers  (T)  is  higher  than  for 
government  feelers  (F) .  On  the 
other  hand,  industry  feelers  (F) 
reported  a  higher  ethical 
sensitivity  score  than  industry 
thinkers  (T) .  The  contradiction 
warrants  further  investigation  into 
the  possible  underlying  causes  for 
this  finding.  This  should  be 
accomplished  by  obtaining  a  larger 
sample  size  and  expanding  the  scope 
of  the  survey  to  include  more 
general  business  practices.  A 
larger  sample  size  will  also  provide 
additional  data  to  examine  the 
relationship  between  ethical 
sensitivity  and  the  sixteen  MBTI 
personality  types. 

National  Contract  Managemant 
Aaaoelation.  Since  this  research 
has  established  a  relationship 
between  ethical  sensitivity  and 
personality  type,  and  because 
further  research  may  expand  these 
findings  and  better  define  their 
implications  and  use,  the  NCMA 
should  make  efforts  to  determine  the 
MBTI  personality  types  of  its 
membership.  An  association-wide 
effort  to  administer  the  survey  and 
make  its  results  part  of  its 
national  database  will  be  useful  not 
only  for  further  research  efforts, 
but  in  the  implementation  of 
recommendations  involving  ethical 
sensitivity  that  will  likely  result 
from  that  research. 

Qovtrnment  and  Industry 
Acquisition  Organization.  The 
relatively  high  ethical 
sensitivities  reported  by  both 
government  and  industry  respondents 
is  an  indication  that  the  effort  to 
increase  ethical  awareness  has  been 
successful,  thus  far.  These 
programs  should  be  continued  and 
expanded  to  capture  the  relatively 


small  percentage  of  respondents  who 
seem  to  still  be  unaware  of  their 
existence . 

Government  and  industry 
acquisition  organizations  are  also 
advised  to  determine  the  MBTI 
personality  types  of  their 
employees.  By  identifying 
personality  types,  managers  can 
conduct  their  own  research  into  the 
attitudes  and  influences  of  these 
types  about  ethics  and  ethical 
awareness  and  behavior.  This  may 
help  to  gain  insights  into  ways  to 
improve  ethics  programs  to  increase 
the  ethical  sensitivities  of  all 
personality  types.  As  the  body  of 
knowledge  involving  ethical 
sensitivity  and  personality  type 
increases,  MBTI  personality  type 
information  will  become  more 
valuable  and  useful. 


Association  for  Psychological 
Type  (APT).  This  research  suggests 
a  rich  area  of  further  study 
involving  the  Myers-Briggs  Type 
Indicator.  The  APT  should  seek-out 
and  sponsor  initiatives  which  expand 
the  scope  of  ethical  sensitivity 
research  and  the  relationships 
between  ethical  sensitivity  and 
personality  type.  Wherever  the  MBTI 
can  be  used  to  expand  the  body  of 
knowledge  on  a  particular  subject 
and,  ultimately,  improve 
interpersonal  understandings  and 
behaviors,  it  not  only  benefits  the 
particular  subject,  it  also  benefits 
society  as  it  is  affected  by  that 
subject.  It  also  increases  the 
prestige  and  reputation  of  the  MBTI 
model  and  type  indicator.  In  this 
case,  there  exists  the  opportunity 
to  further  understand  ethical 
attitudes  and  behaviors  as  they 
relate  to  personality  type,  and 
improve  ethical  behavior  by 
developing  ways  to  best  influence 
different  personality  types  about 
ethics . 
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a  starring  role  to  play. 


ABSTRACT 

Recycling  and  the  preservation  of 
our  natural  resources  is  becoming 
increasingly  important  to  millions 
of  Americans.  Solid  waste  recovery 
is  consistent  with  our  standard  of 
corporate  responsibility  of  help¬ 
ing  to  improve  the  quality  of  life 
in  neighborhoods,  communities  and 
municipalities  nationwide.  Recy¬ 
cling  programs  demonstrate  that  we 
can  effectively  solve  the  solid 
waste  disposal  dilemma.  Changing 
behavioral  characteristics  of  our 
employees  toward  resource  recovery 
IS  necessary  in  order  to  accomo¬ 
date  the  gradual  change  for  future 
generations.  Goals  are  achieved  by 
improving  sanitation  technology, 
implementation  and  enforcement  of 
sensible  waste  handling  regula¬ 
tions,  and  our  continuous  efforts 
in  public  education.  This  whole 
throwaway  approach  flies  in  the 
face  of  traditional  American  values 
of  thrift  and  common  sense. 
Already  costing  us  more  than  $10 
billion  a  year,  trash  disposal  is 
only  getting  more  expensive. 
Other  industrial  countries  produce 
half  as  much  trash  per  person  as  we 
do,  and  recycle  a  major  portion  of 
it.  Recycling  offers  America  a  way 
to  deal  with  trash  that  is  too  good 
to  pa.ss  up.  Creating  less  waste 
and  recycling  more  are  clearly  the 
most  sensible  steps,  followed  by 
safe  incineration  and  landfilling. 
No  single  method  promises  a 
complete  solution  to  the  trash 
crisis,  but  recycling  certainly  has 


INTHODDCTIOH 

Efficient  implementation  of  our 
scrap  recycling  program  is  one  of 
the  most  important  challenges  now 
confronting  military  installations 
today.  The  primary  objectives  of 
these  programs  are  to  conserve 
resources,  minimize  environ.mental 
pollution,  recover  strategic 
critical  materials,  maximize  the 
net  dollar  return  to  the  Federal 
Government  and  improve  the  quality 
of  life  programs  for  everyone.  ‘ 
The  Solid  Waste  Disposal  Act  of 
1965,  as  amended,  requires  that 
federal  facilities  comply  with  all 
federal,  state,  interstate  and 
local  requirements  concerning  the 
management  and  disposal  of  solid 
waste.  Such  requirements  include 
permitting,  licensing,  and 
reporting.  The  Military  Construc¬ 
tion  Codification  Act  (Public  Law 
97-214),  which  became  effective  1 
October  1982,  expanded  the 
Department  of  Defense  (DOD) 
recycling  program  and  required 
significant  changes  in  DOD  policies 
on  the  distribution  of  proceeds 
from  sales  of  recyclable  materials. 
DOD  recycling  programs  pursuant  to 
Section  2577  of  Title  10,  United 
States  Code  provide  increased 
incentives  for  military 
installations  to  establish  and 
operate  efficiently  recycling 
programs.  By  establishing  a  re¬ 
source  recovery  program,  installa¬ 
tions  may  receive  100  percent  of 
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the  proceeds  from  the  sale  of  waste 
materials  and  may  accumulate  up  to 
$2  million  from  waste  sales  which 
may  be  spent  on  pollution 
abatement,  energy  conservation,  and 
occupational  safety  and  health 
deficiency  abatement  projects.^ 

TEXT 

Voluntary  recycling  has  grown  into 
an  important  part  of  our  nation's 
economy.  It  has  given  thousands  of 
entrepreneurs  an  opportunity  to 
start  profitable  and  rewarding 
businesses.  And.  it  provides  tens 
of  millions  of  dollars  in  added 
income  to  the  recycling  public  and 
creates  countless  employment 
positions  nationwide.  The  recov¬ 
ery  effort  to  organize  operations 
that  require  concerted  efforts  by 
concerned  individuals  to  divert  or 
recover  scrap  or  waste  from  waste 
streams,  as  well  as  efforts  to 
identify,  collect,  properly  segre¬ 
gate,  and  maintain  the  integrity  of 
the  recyclable  materials  in  order 
to  .maintain  or  enhance  the 
marketability  of  the  material 
are  steps  in  the  right  direction 
for  all.  For  many,  recovery  of 
recyclable  waste  will  never  warrant 
any  concer.n  until  it’s  too  late.  A 
functional  program  is  not  made  up 
of  one  person  doing  a  lot  but 
rather  a  lot  of  people  doing  a 
little.  Only  than  will  a  recovery 
effort  become  feasible. 

More  so  today  than  ever  before  are 
individuals  becoming  aware  of 
landfill  closures,  illegal  dis¬ 
cards  lapping  up  on  our  beaches 
with  each  incoming  tide,  barges  of 
infectious  waste  traveling  cur 
coast  line  looking  for  a  suitable 
dump  area  with  prior  coordination, 
and,  in  some  instances,  without  any 
coordination.  Behavior  character¬ 
istics  of  our  employees  toward  re¬ 
source  conservat  1  in  a’^e  necessary 
to  create  solutions  and  alter¬ 
natives  in  our  changing  world. 


Growth  in  our  industry  today  is 
riddled  with  an  increase  in  waste 
spills,  ozone  layer  deterioration 
and  the  lack  of  concern  to  do  what 
is  right  for  the  betterment  of 
others.  No  longer  can  we  as  a 
people  continue  to  bury  the 
objectionable  in  landfills,  burn 
the  undesirables  in  incinerators, 
or  otherwise  dispose  of  material  in 
a  manner  harmful  to  the  environment 
and  wasteful  of  our  natural 
resources.  Only  positive  attitudes 
in  a  positive  environment  regarding 
a  team  effort  will  create  change  in 
areas  where  change  is  difficult. 
Commit.ments  to  the  quality  of  air 
we  breathe,  commitments  to  educa¬ 
tion,  commitments  to  a  quality 
workforce,  and  commitments  to  the 
removal  of  recyclable  waste  from 
the  waste  stream  are  no  longer  to 
be  taken  lightly  but  are 
necessary  to  preserve  what  we  have 
grown  to  enjoy  in  this  country. 

Expenses  of  operating  and  improv¬ 
ing  recycling  progra.ms  must  be  ac¬ 
cumulated  and  reimbursed  from 
proceeds  of  sales  of  recyclables 
prior  to  any  other  disposition  of 
revenue.  The  recovery  effort  by 
government  installations  cannot 
operate  efficiently  if  reimburse¬ 
ment  of  revenue  to  the  generator  is 
not  done  in  a  timely  manner. 
Proceeds  from  the  sale  of 
recoverable  waste  at  government 
installations  with  qualifying  re¬ 
cycling  programs  shall  be  deposited 
in  a  special  accounts  to  be  so 
designated  'Budget  Clearing 
Accounts'.  The  accumulation  of 
funds  in  this  suspense  account  is 
not  affected  by  fiscal  year  end.  so 
proceeds  acquired  during  one  fiscal 
year  may  be  carried  forward  and 
merged  with  proceeds  of  subsequent 
fiscal  years.  The  proceeds  shall 
be  segregated  with  the  account  to 
allow  accounting  as  to  the  amounts 
collected  and  their  disposition. 
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The  following  items  are  of 
significant  importance:  1)  Pro¬ 
ceeds  shall  first  be  withdrawn 
from  the  suspense  account  to  cover 
all  the  costs  of  operations, 
maintenance,  and  overhead  for  pro¬ 
cessing  and  handling  the  recy¬ 
clable  materials  (including  the 
cost  of  any  equipment  purchased  for 
recycling  purposes).  2)  If  the 
balance  of  an  installation’s 
proceeds  remaining  in  the  suspense 
account  exceeds  $2  million  at  the 
end  of  a  fiscal  year,  the  amount  in 
excess  of  S2  million  must  be 
deposited  into  the  U.S.  Treasury  as 
miscellaneous  receipts.’ 

Waste  products  which  are  collected 
and  stored  on  a  government  in¬ 
stallation  become  government  pro¬ 
perty  and  must  be  disposed  of 
through  the  services  of  the  Defense 
Reutilization  and  Marketing  Office 
(DRMO) .  Procedures  for  the 
disposal  .must  comply  with  those  as 
outlined  in  the  Defense  Disposal 
Manual  DOD  4160.21M,  The  DRMO  of 
the  Defense  Logistics  Agency  (DLA) 
IS  responsible  for  ne.gotiating 
sales  of  waste  material  from 
military  installations.  In  the 
past,  the  generating  activity  re¬ 
ceived  a  percentage  (about  60/ii  of 
the  proceeds  fro.m  the  sale  of 
recyclable  materials.  The  new 
rules  stipi.ilate  that  all  proceeds 
be  returned  to  installations  with 
an  established  Qualified  Recycling 
Program  (QRF?  .  The  QRF  is  one 
organized  operation  that  requires 
concerted  efforts  to  divert  or 
recover  scrap  or  waste  from  waste 
streams,  as  well  as  efforts  to 
identify,  segregate,  and  maintain 
the  integrity  of  the  recyclable 
materials  in  order  to  maintain  or 
enhance  the  marketability  o;  the 
materials.  Activities  having  pri¬ 
or  agreements  with  DLA/DRMO  for 
waste  sales  must  comply  with  the 
new  provisions  of  ?L  97-214  to 
receive  100  percent  of  the  sales 
proceeds.'*  Military  instal  lations 


not  located  in  the  same  vicinity 
or  far  removed  from  a  servicing 
DRMO  are  encouraged  to  participate 
in  a  waste  recovery  program. 
However,  because  of  their  geo¬ 
graphic  location,  instances  have 
surfaced  which  deter  the  probabili¬ 
ty  of  success.  Numerous  publica¬ 
tions  outline  the  disposition  of 
recoverable  waste  products  from 
government  installations  and  often 
outline  the  merits  of  a  waste  to 
energy  program.  Although  adequate 
when  initially  written,  times  have 
changed.  The  recovery  effort  is 
mere  defined  and  not  all  installa¬ 
tions  can  p.articipate .  DRMO’s  are 
burdened  with  the  disposal  of  ser¬ 
viceable  products  or  products  which 
can  be  classified  as  those  which 
can  be  used  for  their  intended 
purpose.  Products  of  this  nature 
are  not  considered  recyclable. 
Priority  has  shifted  from  and 
rightfully  so,  to  the  disposal  of 
the  items  which  can  be  moved  most 
rapidly  from  the  staging  areas. 
Normally,  miscellaneous  scrap  does 
not  move  rapidly  and  the 
accumulation  beco:r.es  unattractive 
to  the  general  appearance  of 
onlooking  .management  personnel.  It 
needs  to  be  stockpiled  into  l-ots, 
aIlov;ed  to  accumulate  in  sufficient 
quantities  to  increase  it’s  value, 
and  source  separated  to  increase 
It’s  integrity.  All  of  this  re¬ 
quires  a  concerted  effort  from 
those  individuals  directly  involved 
in  the  process. 

Change  for  the  future  is  necessary 
to  remove  waste  from  the  waste 
stream.  Individuals  participate 
when  the  procedure  is  comfortable, 
paperwork  is  minimized  and  profits 
realized.  Instances  do  arise  where 
the  servicing  DRMO  cannot  handle 
the  sale  and  disposition  of  mer¬ 
chandise,  which  has  been  collected 
and  stored  in  a  warehouse,  in  time 
frames  commensur.ate  with  the 
collection.  Established  proce- 
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dures  for  the  sale  of  recyclable 
waste  do  take  time.  Material  in¬ 
spection,  contract  preparation  and 
advertising  all  lengthen  the  pro¬ 
cess.  Products  not  brought  to 
markets  quickly  lose  their  value, 
storage  space  diminishes,  and 
continuity  in  program  operation 
along  with  the  integrity  of  the 
products  decrease. 

The  provisions  in  appropriate 
government  directives  need  to 
accommodate  situations  which  may  be 
beneficial  to  the  government  and 
assist  in  the  reduction  of  the 
enormous  amounts  of  waste  in  the 
waste  stream.  Revised  and/or 
implement  procedures  for  the 
stockpiling  of  recyclable  waste  in 
designated  locations  after  natural 
disasters  in  most  cases  has  not 
been  addressed.  For  example,  due 
to  the  destruction  by  hurricanes, 
fires,  earthquakes,  floods  and 
other  disasters,  there  are  recy¬ 
clable  wastes  generated  in  great 
quantities.  These  .may  include 
aluminum  and  other  metals, 
aircraft,  and  timbers.  Present 
procedures  in  most  instances  only 
include  removal  and  disposal  by 
burning  and/or  burying  in  land¬ 
fills.  After  the  emergency  has 
come  under  control  in  terms  of 
human  suffering,  and  the 
restoration  process  begins,  it 
would  be  profitable  to  the 
government  to  have  emergency  recy¬ 
cling  procedures.  Perhaps  this 
could  be  in  the  form  of  special 
teams  that  recover  and  transport 
materials  to  designated  areas.  The 
net  gain  can  only  be  beneficial,  to 
include  monetary  recovery.  Our 
present  discarding  sometimes  only 
allows  for  partial  recovery  by 
private  individuals  or  companies. 
It  IS  time  to  implement  local 
operating  procedures  to  sell 
certain  scrap  materials  due  to 
insufficient  quantities  and/or  in 
in  the  absence  of  a  recycling  pro¬ 
gram  or  servicing  DRMO.  Other¬ 


wise,  recoverable  waste  will  be 
disposed  of  as  municipal  trash  and 
garbage:  i.e.,  waste  paper,  cor¬ 
rugated,  cullet,  lumber,  rags, 
bones  and  fats,  grease,  wood 
scrap,  batteries,  pallets,  rubber 
scrap,  plastics,  tile,  concrete, 
bricks  and  clay,  ferrous/non- 
ferrous  metals  if  no  contract 
exists,  containers,  and  other 
material  too  numerous  to  mention. 

Market  sources  available  for  the 
sale  of  recyciables  collected  by 
our  government  locations  are  not 
always  exploited  down  to  the  local 
level  by  the  DRMO.  Profitable 
local  markets  for  recyciables  are 
frequently  unused  when  limited 
quantities  or  a  market  price 
decline  prevents  a  sale  on  a 
regional  basis.  Also,  disposal 
through  the  local  DRMO  may  prove 
impossible  if  adequate  storage 
volume  IS  limited  at  that  site  and 
prevents  the  accumulation  of  the 
large  quantities  needed  to  made 
DEMO  sales  contracts  attractive  to 
bidders.  These  conditions  release 
the  DRMO  from  their  obligation  to 
accomplish  the  sale  of  the  product 
for  the  government  when  bidders  do 
not  respond.” 

COHCLUSIOHS 

Inefficient  waste  management 
practices  used  in  the  past,  such  as 
burning  in  open  dumps  are  no 
longer  environmentally  or 
economically  acceptable.  Problems 
in  today’s  industrialized  society, 
such  as  the  growing  volume  of  solid 
waste,  air,  water  and  land 
pollution.  and  rising  materials 
cost  are  being  countered  with 
programs  for  waste  reduction, 
stringent  regulations  for  environ¬ 
mental  protection,  and  require¬ 
ments  for  recycling  and  resource 
recovery.  Cost  ef f ecti veness  of 
recycling  and  resource  recovery 
purports  that  the  net  cost  of  such 
operations  will  be  equal  to  or  less 
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than  that  of  conventional 
incineration  or  sanitary  landfill 
disposal.  Today,  the  environment 
is  considered  to  be  a  single, 
integrated  system  characterized  by 
the  continuous  interaction  of  air. 
land  and  water.  This  system  should 
be  regarded  as  closed;  nothing  can 
be  thrown  away.  Wastes  must  either 
be  recycled,  reclaimed,  repaired  or 
confined  and  contained  so  they  will 
not  reappear  in  a  pollutant  form.* 

As  managers  of  the  public  trust, 
the  government  would  be  derelict  in 
it’s  public  trust  responsibility  to 
allow  such  exigencies  of  regional 
markets  to  prevent  capitalization 
of  all  recyclable  refuse  resources, 
even  if  on  a  modest  local  scale. 
Considering  the  fact  that  some 
strategic  metals  are  not  found  in 
North  America,  above-ground  mining 
in  salvage  yards  may  be  the  only 
source  of  procurement  during 
periods  of  hostility.  Metals 
which  can  withstand  high  tempera¬ 
tures  are  necessary  in  the 
manufacturing  of  critical 
components  of  the  weapons  of 
tomorrow,  i.e.,  engine  parts, 
guidance  systems,  etc.  Addition¬ 
ally,  we  have  an  obligation  to  the 
environment  and  to  our  limited 
natural  raw  material  resources  to 
return  expended  products  to  the 
waste  s'^ream,  so  long  as  the 
activity  is  self-supporting.  When 
a  regional  or  area  buyer  cannot  be 
found  to  profitably  serve  a 
specific  location,  the  installa¬ 
tion  then  has  products  on  hand 
which  the  DRMO  is  not  obligated  to 
accept.  The  installation  then 
faces  a  cost  for  disposal  rather 
than  a  profit  situation.  Costs  to 
the  government  can  be  reduced  in 
this  area  alone,  by  not  incurring 
an  additional  disposal  cost  for  a 
recoverable  product.’’  • 


outcome  is  in  the  best  interest  of 
the  government.  This  would  allow 
profitable  sales  to  smaller  sal¬ 
vage  companies  in  the  commuting 
vicinity  of  a  military  or 
government  installation. 

Based  on  the  existence  of  the 
above  conditions,  individual  gov¬ 
ernment  locations  need  the  author¬ 
ization  to  search,  exploit  and 
utilize  localized  markets  for  sale 
of  waste  products  without  using  the 
administrative  and/or  sales 
structure  of  the  DRMO  system.  To 
comply  with  current  and  future 
requirements,  innovative  manage¬ 
ment  is  needed  at  all  levels  of 
responsibility  to  close  gaps  and 
create  change.  Enable  commanders 
latitude  and  flexibility  to  estab¬ 
lish  markets,  implement  local 
operating  instructions  and  deter¬ 
mine  what  IS  best  for  their 
installation.  This  quantum  change 
will  foster  ones  imagination  and  is 
innovative.  Implementation  will 
strengthen  the  defense  industrial¬ 
ized  base. 

MOTES 

^  'To  Defense  Logistics 
Agency  Personnel’,  14  Mar  85, 
Selected  Letter  of  Brigadier  Gen¬ 
eral  Thomas  B.  Arwood.  (Commander: 
Defense  Property  Disposal  Service 
Battle  Creek,  Michigan,  Commander's 
Guidance  Statement  (CGSl  NO:  21. 

-  Laws  of  the  7th  Congress 
2nd.  Se-ssion,  F.L.  97-214,  July 
1982,  96  STAT.  172. 


We  should  bypass  the  knot  tying 
procurement  procedures  of  adver¬ 
tising  and  vendor  chase  if  the 
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83. 

“  "USAF  Moral  Welfare,  and 
Recreation,"  Guide  For  Developing  A 
Resource.  Recovery,  And  Recycling 
Program.  HQ  AFMPC/DPMS,  Randolph 
AFB.  Tx. 

’  Letter  received  from 
Chuck  West.  HQ  AFESC/DEM.  Trash  and 
Waste  Recycling  (TWR) .  12  Jun  1985. 

®  Army  PAM  420-47,  Solid 
Waste  Management.  June  1978,  p. 
1-2. 

Letter  received  from 
Chuck  West,  HQ  AFESC/DEM,  Sales  of 
Recyclable  Materials  (10  USC  2577), 
18  Jun  1985. 
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ACfflEVING  PUBLIC  ENDS  WITH  PRIVATE  MEANS: 

THE  GOVERNMENT’S  ACQUISITION  OF  R&D  FROM  FFRDCs 
Douglas  L.  Sanders,  Jet  Propulsion  Laboratory 


ABSTRACT 

Federally  funded  research  and  development 
centers  (FFRDCs)  are  private,  nonprofit 
organizations  that  perform  R&D  almost 
exclusively  for  a  single  federal  agency.  They 
often  look  like  public  organizations  because  they 
work  closely  with  their  sponsoring  agencies  and 
have  "insider"  access  to  information  on  agency 
plans  and  programs.  Many  in  the  public  sector 
think  FFFUDCs  have  too  much  influence  over 
agency  decisions  and  circumvent  civil  service 
and  procurement  regulations.  The  private  sector 
criticizes  FFRDCs  because  they  do  not  compete 
for  work  received  and  are  supposedly  inefficient 
producers.  This  paper  uses  a  historical 
perspective  to  illuminate  the  current  policy 
issues  surrounding  FFRDCs,  including  the 
policy  implications  of  FFRDCs  as  potential 
competitors  with  other  research  and  development 
(R&D)  performers  and  the  advisability  of 
converting  other  performers  to  FFRDCs.  The 
paper  concludes  that  FFRDCs  fill  an  important 
niche  as  performers  of  R&D  that  cannot  be 
easily  fill^  by  others. 

INTRODUCTION 

When  Dr.  J.  Robert  Oppenheimer  was 
selected  by  Major  General  Leslie  Groves  in  the 
fail  of  1942  to  administer  a  classified 
laboratory  at  Los  Alamos,  N.  M.,  both  men 
assumed  the  facility  would  follow  conventional 
practice  and  become  militarized.  Given  the 
source  of  funds,  the  lab’s  mission  to  develop  the 
first  atomic  bomb,  and  the  need  for  security,  it 
was  assumed  that  scientists  and  engineers  would 
be  conunissioned  as  army  officers.  But  several 
key  scientists  refused  to  Join  the  Los  Alamos 
team  under  conditions  of  military  hierarchy  and 
bureaucracy.  Science,  they  believed,  thrived 
when  scientists  were  free  of  bureaucratic 
constraints  and  Judged  according  to  their 
competence.’ 

In  a  February  25,  1943,  letter  to 
Oppenheimer,  General  Groves  relented,  promis¬ 


ing  that  Los  Alamos  would  be  a  civilian 
operation  managed  by  the  University  of 
California  under  an  Army  contract.  The  Groves 
letter  was  the  first  deliberate  step  in  legitimizing 
a  new  type  of  institution  that  came  to  be  known 
as  the  federally  funded  research  and 
development  center.  It  also  explicitly  recognized 
that  the  type  of  institution  performing  R&D 
matters.  Even  General  Groves  came  to 
recognize  that  this  new  approach  worked  far 
better  than  the  militarized  arrangement.  While 
the  scientists  at  Los  Alamos  were  concerned 
about  freedom  of  scientific  inquiry  in  a  military 
setting,  the  concern  has  implications  for 
scientific  inquiry  on  the  civilian  side  as  well. 
This  is  Just  one  issue  that  has  arisen  in  trying  to 
accomplish  public  ends  with  private  means. 

It  is  somewhat  paradoxical  that 
democracy  and  science  flourish  in  the  same 
libertarian  climate.  As  former  Caltech  president 
Lee  DuBridge  noted  almost  thirty  years  ago,  "in 
a  republic  many  people  are  concerned  with 
government.  On  the  other  hand  few  people  are 
directly  concerned  with  science.  How  can  the 
many  be  made  to  understand  the  contributions 
and  the  problems  of  the  very  few?  This  is  a 
major  problem  in  any  democracy. 

Using  a  methodological  approach  that  is 
essentially  historical,  this  article  attempts  to 
show  that  the  FFRDC  concept  is  an  effective 
means  of  reconciling  the  needs  of  the  few  to  the 
legitimate  concerns  of  the  many,  while 
harnessing  the  talents  of  the  few  to  the  service  of 
the  many  as  represented  by  the  federal 
government.  The  approach  lends  itself  to 
answering  some  key  questions  about  FFRDCs, 
including;  Why  and  how  were  they  created? 
What  factors  contributed  to  their  institutional 
design?  What  advantages  and  disadvantages  do 
they  have  as  performers  of  R&D?  How  are  they 
different  from  profit-making  contractors?  What 
caused  the  later  conflicts  between  FFRDCs  and 
other  institutions  performing  federal  R&D?  And 
perhaps  most  importantly,  should  the  FFRDC 
model  be  expanded  to  other  government-funded 
R&D  efforts?  Tnis  paper  looks  primarily  at 
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research,  multi-program,  and  studies  and 
analysis  FFRDCs  that  have  discretion  to  initiate 
and  manage  elements  of  their  own  R&D 
programs.  Other  FFRDCs,  more  appropriately 
characterized  as  government-owned,  contractor- 
operated  (COCO)  facilities,  are  not  the  primary 
focus. 

DEFINING  FFRDCs 

What  exactly  are  FFRDCs?  The 
National  Science  Foundation  (NSF)  defines  them 
as  "R&D  performing  organizations  exclusively 
or  substantially  financed  by  the  federal  gov¬ 
ernment  .  .  .  either  to  meet  a  particular  R&D 
objective  or,  in  some  instances  to  provide  major 
facilities  at  universities  for  research  and  assoc¬ 
iated  training  purposes.  Each  center  is 
administered  either  by  an  industrial  firm,  a 
university,  or  another  nonprofit  institution."® 
The  following  criteria  must  be  met  before  the 
NSF  includes  an  organization  in  the  FFRDC 
category: 

1 .  Its  primary  activities  include  R&D, 
management  of  R&D,  or  a  combination  of  these 
activities; 

2.  It  is  a  separate  operational  unit 
within  a  parent  organization  or  a  separately 
incorporated  organization; 

3.  It  performs  work  either  at  the 
government’s  request  or  under  a  broad 
government  charter,  but  always  under  direct 
government  monitorship; 

4.  It  receives  most  financial  support 
from  one  agency; 

5.  It  has  a  long-term  relationship  with 
its  sponsoring  agency  as  evidenced  by  specific 
obligations  assumed  by  each; 

6.  Most  or  ail  facilities  are  owned  by, 
or  are  funded  under  contract  with,  the 
government;  and 

7.  It  has  an  average  annual  budget  of  at 
least  $500,000.“ 

These  criteria  do  not  clearly  distinguish 
FFRDCs  from  private,  profit-making 
government  contractors,  particularly  certain  cost- 
type  defense  contractors.  Only  criterion  5. 
somewhat  distinguishes  FFRDCs  from 
government  contractors,  and  even  it  could  be 
interpreted  as  applying  to  profit-making 


contractors.  Indeed,  it  may  not  be  possible  to 
clearly  distinguish  between  FFRDCs  and  their 
profit-making  brethren.  The  only  other  plausible 
distinguishing  factors  relate  to  the  market 
environment  in  which  profit-making  contractors 
supposedly  operate.  More  on  this  below. 

THE  EARLY  FFRDC  EXPERIENCE 

Los  Alamos  was  not  the  first  FFRDC- 
like  arrangement.  That  honor  should  probably 
go  to  the  Applied  Physics  Laboratory  at  Johns 
Hopkins  University,  which  was  organized  in 
1942  at  the  request  of  the  Office  of  Scientific 
Research  and  Development  (OSRD).®  The 
laboratory  directed  an  association  of  universities 
and  industrial  contractors  developing  new 
concepts  for  weapons  systems.®  Other  wartime 
FFRDCs  included  Harvard’s  Underwater  Sound 
Laboratory,  the  M.I.T.  Radiation  Laboratory, 
and  the  Jet  Propulsion  Laboratory  (JPL)  at 
Caltech.  JPL,  started  by  the  Army  in  1944 
under  a  contract  with  Caltech,  was  transferred  to 
the  National  Aeronautics  and  Space 
Administration  (NASA)  in  1959.® 

After  World  War  II  FFRDCs  flourished. 
Several  agencies,  including  the  NSF,  Atomic 
Energy  Commission  (AEC),  Department  of 
Defense  (DOD),  and  NASA  expanded  existing 
FFRDCs  or  sponsored  new  ones.  Some 
FFRDCs  were  formed  to  circumvent  the 
"bureaucratic  delay  inherent  in  government."* 
Other  reasons  why  agencies  sought  support  from 
groups  of  outside  experts  via  FFRDCs,  included 

(1)  low  government  pay  scales  for  professionals, 

(2)  a  conscious  desire  to  prevent  agencies  from 
developing  large  permanent  technical  staffs,  (3) 
the  need  for  independent  technical  judgment,  (4) 
the  increasing  scientific  and  technological 
complexity  of  major  new  systems,  and  (5)  the 
specialization  required  to  develop  new,  highly 
complex  systems. 

Building  on  the  wartime  experience,  the 
Atomic  Energy  Act  of  1946  created  the  AEC  to 
produce  weapons-grade  nuclear  materials  for 
DOD,  develop  reactor  technology  for  the  private 
sector,  and  regulate  commercial  reactors.  R&D 
facilities  created  or  assigned  new  responsibilities 
under  the  Manhattan  Project  played  a  critical 
role  in  postwar  science  and  technology  develop- 
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ment.  The  most  important  were  the 
Metallurgical  Laboratory  of  the  University  of 
Chicago,  Ernest  Lawrence’s  Laboratory  at 
Berkeley,  the  Clinton  Engineering  Works  which 
became  the  Oak  Ridge  National  Laboratory,  the 
Hanford  Engineering  Works,  and  the  Los 
Alamos  Scientific  Laboratory. 

The  AEC  deliberately  chose  not  to 
operate  the  laboratories  inherited  from  the 
Manhattan  Project,  even  though  the  Atomic 
Energy  Act  specifically  provided  for  federally- 
conducted  R&D.  In  his  book  Contracting  for 
Atoms.  Harold  Orlans  characterized  this  policy 
as  "self-denying"  and  "remarkable."  However, 
he  offered  several  reasonable  explanations  why 
a  majority  in  Congress  and  the  AEC  did  not 
favor  government  operation  of  major  R&D 
installations: 

1 .  Most  key  indu.strialists  and  scientists 
opposed  it,  because  they  believed  military  or 
civil  service  management  would  be  less  effective 
than  private  management  and  would  make  it 
difficult  to  attract  the  best  people. 

2.  Allowing  private  organizations  to 
manage  government-owned  laboratories  was  the 
only  way  to  ameliorate  AEC’s  "draconian 
powers"  that  abrogated  traditional  rights  of 
private  ownership  and  the  free  exchange  of 
information. 

3.  Contracting  with  private 
organizations  allowed  the  migration  of 
knowledge,  ideas,  and  advice  "from  a  larger  and 
more  distinguished  number  of  industrialists  and 
scientists  than  would  otherwise  have  participated 
in  Commission  affairs."’ 

The  legal  mechanism  for  this 
public/private  endeavor  was  the  "administrative 
contract"  under  which  private  organizations, 
including  universities,  other  non-profit 
corporations,  and  commercial  companies,  agreed 
to  manage  AEC  laboratories  for  the  government. 
Commercial  companies  typically  received  small 
management  fees,  but  also  benefitted  from  the 
experience  their  employees  received  and  the 
transfer  of  new  technology.'”  These  contracts 
envisioned  a  relationship  between  FFRDCs  and 
their  sponsors  that  was  different  from  the 
relationship  for  typical  government 
procurements.  The  latter  were  (and  still  are) 
conducted  on  a  competitive  basis  through  arm’s 


length  bidding  or  negotiating  for  specific 
products  or  services.  The  former  presupposed 
a  long-term,  cooperative  relationship  involving 
continuous  interaction  between  contractors  and 
sponsors  to  determine  what  the  parties  would 
provide.  Although  the  sponsor  determined  an 
FFRDC’s  basic  direction,  approved  its  long- 
range  plans,  and  played  a  role  in  administrative 
matters,  the  FFRDC  managed  the  work. 

In  short,  administrative  contracts  allowed 
the  AEC,  and  other  Government  agencies  later, 
to  capture  a  part  of  the  scientific  community  for 
its  own  purposes,  while  preserving  the 
flexibility,  freedom  of  inquiry,  and  reduced 
bureau-demanded  by  scientists. 

The  Department  of  Defense  (DOD) 
reached  the  same  conclusion  about  the  need  for 
FFRDCs,  but  for  somewhat  different  reasons. 
The  primary  issue  for  DOD  was  how  to  obtain 
unbiased  policy  advice  in  advance  of  agency 
needs.  FFRDCs  (called  Federal  Contract 
Research  Centers  in  DOD)  like  the  RAND 
Corporation  and  the  Institute  for  Defense  Ana¬ 
lyses  (IDA)  were  established  for  this  purpose. 
I^ND’s  charter  became  a  model  for  later 
defense-oriented  FFRDCs  in  its  commitment  "to 
further  and  promote  scientific,  educational  and 
charitable  purposes,  all  for  the  public  welfare  of 
the  United  States  of  America.""  As  a  side 
benefit  sponsoring  defense  agencies  hoped 
FFRDCs  would  be  catalysts  for  innovation 
within  the  agencies. 

RAND  is  engaged  primarily  in  long 
range  research  and  analyses  to  aid  strategic  and 
technical  planning.  Unlike  many  of  the  AEC 
FFRDCs,  RAND  was  not  established  to 
manufacture  hardware  as  deliverable  items  to  the 
government.  This  enhanced  its  image  as  a 
provider  of  "disinterested"  advice,  as  it  had  less 
incentive  to  slant  its  work  product  in  an  effort  to 
obtain  folluw-on  production  work.  RAND  had 
a  direct  hand  in  establishing  two  other  FFRDCs. 
The  Systems  Development  Corporation  (SDC) 
was  spun  off  by  RAND  in  1956  to  help  design 
ai.d  program  the  first  computerized  air  defense 
system.  RAND  also  fostered  Analytic  Services 
Inc.  (ANSER)  to  provide  quick  reaction  analyses 
of  problems  incident  to  planning  for  the  R&D  of 
aerospace  weapon  systems.'" 
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Several  special  purpose  FFRDCs  were 
created  after  World  War  II  to  assist  the  Air 
Force  with  technology  development  thrusts 
initiated  during  the  war.  The  Lincoln 
Laboratories,  a  lineal  descendant  of  the  wartime 
Radiation  Laboratory  at  M.I.T  and  the  MITRE 
Corp.,  which  oversaw  integration  of  the  air 
defense  system.  The  largest  Air  Force  FFRDCs 
was  the  Aerospace  Corporation,  founded  in 
1960.  The  corporation  provided  technical 
direction  and  systems  engineering  for  the  Air 
Force’s  ballistic  missile  and  space  program. 
Before  1960  this  task  was  performed  by  what 
one  congressman  described  as  a  "hardware- 
oriented,  profit-seeking  private  contractor."** 
The  Military  Operations  Subcommittee,  House 
Armed  Services  Committee,  noting  the 
importance  of  disinterested  advice, 
recommended  that  the  Air  Force  shift  this  work 
to  "a  nonprofit  institution  akin  to  the  RAND 
Corp.  and  other  .  .  .  organizations  which  serve 
the  military  departments  and  other  agencies  of 
the  Government  on  a  stable  and  continuing 
basis."** 

The  Air  Force  was  not  alone  in 
sponsoring  FFRDCs  in  the  post-war  years.  The 
Defense  Department  sponsored  IDA  in  1956  to 
conduct  studies  on  the  rationale  of  defense  R&D 
programs  and  on  national  security  policy 
matters.  The  Army  fostered  the  creation  of  the 
Research  Analysis  Corporation  (RAC)  in  1961 
to  conduct  operations  research.  RAC  succeeded 
the  Operations  Research  Office  operated  by 
Johns  Hopkins  University.  In  1962  the  Navy 
established  the  Center  for  Naval  Analysis  (CNA) 
as  a  division  of  the  Franklin  Institute  in 
Philadelphia. 

It  was  anticipated  that  the  need  for  many 
DOD  FFRDCs  would  be  temporary.  However, 
they  became  increasingly  valuable  because  of 
their  success  in  attracting  many  talented 
professionals  with  special  skills  and  expertise  in 
diverse  fields.  This  led  to  a  continuing  flow  of 
new  and  extended  requirements  for  their  special 
services  which  caused  them  to  increase  both  in 
number  and  size.  Increasing  demands  for  the 
services  of  FFRDCs  resulted  in  their  becoming 
sizeable  private  organizations.  Growth  was 
accompanied  by  expansion  and  diversification, 
which  attracted  other  clients.  These  included 


federal  civilian  agencies  and  organizations 
outside  the  government.'* 

FFRDCs  AND  THE  POLICY  PROCESS 

Were  FFRDCs  legitimate  from  a  public 
policy  perspective?  I  have  found  very  few 
references  to  any  of  the  steps  normally 
associated  with  the  public  policy  process.  It 
spears  the  AEC  (and  later  DOD)  identified  the 
problem  Oack  of  a  suitable  institution  to  perform 
certain  types  of  R&D),  formulated  and  adopted 
the  preferred  policy  (creation  of  FFRDCs), 
administered  programs,  and  evaluated  results, 
with  very  little  debate  in  Congress  or  elsewhere. 
This  is  not  exactly  what  one  would  expect  today 
given  the  economic  and  political  implications  of 
establishing  a  new  institution  to  perform  R&D. 
Not  only  was  debate  limited,  but  there  was  tittle 
protest  from  the  other  performers  of  R&D. 

There  are  four  plausible  explanations. 
First,  with  the  advent  of  the  Cold  War,  defense- 
related  scientific  research  was  conducted  in  an 
atmosphere  of  urgency,  if  not  crisis.  DOD  and 
AEC  were  given  great  discretion  in  their  pursuit 
of  defense-related  scientific  objectives.  It  would 
have  been  imprudent  at  best  and  unpatriotic  at 
worst  to  interfere  with  the  exercise  of  that 
discretion.  Second,  government  R&D  on  such 
a  large  scale  was  a  new  phenomenon,  dating 
only  from  World  War  II.  Industrial  and 
university  performers  were  new  to  their  roles  as 
recipients  of  major  and  continuous  defense  fund¬ 
ing.  Perhaps  they  did  not  protest  the  advent  of 
FFRDCs  because  they  did  not  view  themselves 
as  possible  future  competitors  of  FFRDCs  for 
federal  R&D  money.  In  fact,  as  the  parent 
organizations  for  most  of  the  large  FFRDCs, 
universities  anticipated  substantial  benefits  in  the 
form  of  research  opportunities  for  faculty  and 
students,  and  fees  for  operating  the  centers.**  A 
related  explanation  is  that  the  full  plates  of  the 
other  performers  led  to  complacency  about  the 
amount  of  R&D  funding  that  went  to  FFRDCs. 
The  last  explanation  concerns  capacity.  Both 
proponents  and  opponents  of  FFRDCs  agree  that 
neither  the  university  community  nor  industry 
was  capable  of  amassing  the  array  of  expertise 
required  to  meet  certain  specialized  government 
demands  for  R&D  following  World  War  II. 


534 


VISIBILITY  LEADS  TO  CONFLICT 


From  the  late  1940s  to  the  early  1960s 
the  role  of  FFRDCs  does  not  appear  to  have 
been  controversial,  even  though  they  were 
receiving  16%  of  federal  R&D  obligations  by 
1955  (Table  1).  The  issue  of  which  institution^ 
performers  should  get  what  in  terms  of  R&D 
dollars  was  addressed  in  the  1962  "Bell  Report” 
to  the  President.  The  report  was  authored  by 
the  heads  of  DOD,  AEC,  NASA,  NSF,  and  the 
Civil  Service  Commission.  It  concluded  that, 
"in  selecting  recipients,  whether  public  or 
private,  for  research  and  development 
assignments,  the  basic  rule  .  .  .  should  be  to 
assign  the  job  where  it  can  be  done  most 
effectively. 

In  1967,  the  Federal  Council  for  Science 
and  Technology  prescribed  criteria  for 
identifying  FFRDCs  and  developed  an  official 
master  list.  In  1971,  NSF  listed  70  FFRDCs; 
27  were  small  educational  laboratories  funded  by 
the  Office  of  Education,'*  the  AEC  accounted 
for  21,  DOD  sponsored  15,  and  7  were 
sponsored  by  other  agencies. These  early 
lists,  by  establishing  FFRDCs  as  a  separate 
category  of  R&D  performer,  called  attention  to 
their  institutional  character-something  that  had 
not  been  done  when  they  were  established. 
Budget  data  generated  as  a  natural  supplement  to 
the  lists  was  instrumental  in  highlighting  for 
Congress,  the  public,  and  other  performers  the 
share  of  R&D  funding  that  went  to  FFRDCs. 

The  public  policy  debate  over  the 
institutional  character  and  role  of  FFRDCs, 
which  did  not  occur  in  the  1940s  and  1950s, 
finally  took  place  in  the  late  1960s  and  early 
1970s.  Criticism  of  FFRDCs  came  from 
Congress,  industry,  the  print  media,  and 
academia.  Even  though  the  FFRDC  share  of 
federal  R&D  obligations  had  dropped  to  nine 
percent,  critics  were  concerned  that  FFRDCs  (I) 
were  relatively  free  from  congressional  control, 
(2)  had  too  much  influence  over  DOD  policy 
making,  (3)  were  a  subterfuge  to  avoid  the 
restrictions  of  civil  service  salary  scales,  (4)  had 
outlived  the  rationale  for  their  creation,  (5)  were 
not  subject  to  normal  restraints  of  the 
competitive  marketplace,  and  (6)  were  exempt 
from  the  restrictions  of  government  regulations. 


Where  universities  were  parent  organizations  of 
FFRDCs,  the  complaint  centered  on  university 
support  for  classified  military  research.  Campus 
protests  were  an  important  factor  in  the 
decisions  of  M.I.T.  and  Columbia  to  change 
their  relationships  with  Lincoln  Laboratories  and 
IDA,  respectively. 

Issue  (2)  is  of  particular  interest.  The 
influential  advisory  role  of  some  FFRDCs,  such 
as  RAND  and  MITRE,  caused  some  in  Congress 
to  fear  that  decision-making  responsibility, 
which  they  saw  as  the  duty  and  privilege  of 
government  officials,  was  being  contracted  out. 
This  concern  is  similar  to  Frank  Fischer’s 
concerns  about  technocracy,  which  he  defines  as 
"a  system  of  government  in  which  technically 
trained  experts  rule  by  virtue  of  their  specialized 
knowledge  and  position  in  dominant  political  and 
economic  institutions.  Perhaps 

congressional  concern  over  the  influence  of 
FFRDCs  on  DOD  policy  making  was  a 
precursor  of  the  technocratic  concerns  many 
have  today. 

By  the  end  of  1971,  the  House 
Committee  on  Appropriations  concluded  it  was 
time  for  a  change  in  philosophy  regarding  the 
use  of  DOD  FFRDCs.  The  committee  directed 
DOD  to  reduce  its  use  of  RAND,  RAC,  CNA, 
and  IDA,  and  recommended  cutbacks  of  25%  in 
the  budget  requests  for  each  FFRDC  in  fiscal 
year  1972.  The  committee  also  indicated  it 
expected  further  cutbacks  in  future  years 

However,  in  response  to  pressures  from 
FFRDC  parent  organizations  (mostly 
universities)  and  Congress,  DOD  had  already 
begun  to  decrease  the  number  of  FFRDCs  it 
sponsored.  In  the  early  1960s  DOD  sponsored 
39  FFRDCs. “  By  1967  that  number  had 
dropped  to  18  and  by  1972  it  decreased  to  12. 
In  1976  only  6  DOD  FFRDCs  remained.  Some 
of  the  organizations  struck  from  the  list  of 
FFRDCs  included  the  Applied  Physics 
Laboratory  at  Johns  Hopkins,  the  Advanced 
Research  Laboratory  at  Penn  State,  and 
ANSER.®  Also,  the  number  of  professional 
staff  members  employed  by  the  three  remaining 
FFRDCs  the  committee  was  most  concerned 
about  decreased  from  975  in  1967  to  about  660 
in  1975.  Even  with  the  reductions,  however, 
DOD  continued  to  consider  these  studies  and 


Table  1 

FFRDC  Share  of  Federal  Obligations  for  R&D  (Billions 


1955 

I960 

1965 

1970 

1975 

19?0 

1985 

1990 

FFRDCs: 

.4 

.8 

1.2 

1.4 

1.9 

3.4 

5.3 

5.9 

Total: 

2.5 

7.6 

14.6 

15.3 

19.0 

29.8 

48.4 

62.3 

FFRDC  %: 

16% 

11% 

8% 

9% 

10% 

11% 

11% 

9% 

analysis  centers  their  best  source  of  high- 
quality,  independent  professional  judgement  on 
various  policy  issues. “ 

The  FFRDC  controversy  may  appear  to 
have  been  a  tempest  in  a  teapot  given  the  small 
part  of  the  federal  R&D  budget  that  went  to  the 
centers  during  that  period  (see  Table  1). 
However,  FFRDCs  directly  or  indirectly 
influenced  a  much  larger  share  of  expenditures 
for  R&D  and  hardware  procurements. 
Moreover,  they  had  a  disproportionately  great 
influence  on  U.S.  R&D  achievements  because  of 
their  concentration  on  missions  of  national 
significance.  Almost  all  industrial  organizations 
contracting  with  DOD  or  AEC  were  affected  by 
the  results  of  FFRDC  studies.  In  effect, 
FFRDCs  had  some  of  the  characteristics  of  a 
super  technocracy-a  technocracy’s  technocracy— 
that  could  exercise  a  degree  of  control  over  the 
fortunes  of  organizations  in  the  ordinary 
technocracies.  It  is  not  surprising,  therefore, 
that  congressional  interest  in  FFRDCs  was 
piqued,  in  part,  by  companies  who  resented 
working  under  the  close  technical  direction  of 
certain  FFRDCs. “ 

FRAMING  THE  ISSUE 

The  political  debate  over  DOD’s 
FFRDCs  had  some  positive  effects.  From  a 
public  policy  perspective  it  helped  focus  the 
advantages  and  disadvantages  of  FFRDCs  as 
performers  of  R&D.  A  study  funded  by  NSF 
and  conducted  by  the  University  of  Denver 
Research  Institute  was  particularly  useful.^’  It 
identified  several  advantages  and  disadvantages 
to  government  use  of  FFRDCs,  all  of  which 
(plus  others)  are  still  relevant  today: 


Advantages 

1.  Relative  ease,  when  compared  to 
government  agencies,  in  attracting  and 
retaining  high-quality  staff  members.  Under 
current  civil  service  rules,  the  government  still 
has  problems  attracting  the  quantity  and  quality 
of  scientists  and  engineers  it  requires.^ 

2.  High  objectivity  of  R&D  output. 
Those  who  hold  this  view  note  that  FFRDCs  are 
organizationally  separate,  receive  fairly  constant 
funding  and,  do  not  usually  compete  for  the 
work  they  receive. 

3.  High  quality  of  R&D  work.  This 
claim  is  disputed  by  some  industrial  performers, 
but  the  often  spectacular  successes  of  FFRDCs- 
such  as  RAND  in  systems  analysis  and  JPL  in 
interplanetary  exploration-are  well  documented. 

4.  Freedom  from  conflicts  of  mtcrcot 
to  which  industrial  performers  of  R&D  are 
susceptible.  Conflicts  of  interest  (COIs)  are 
often  related  to  competition  and  the  profit 
motive;  e.g.,  a  firm  competing  for  a  cost-type 
contract  may  give  a  biased  projection  of  the  cost 
and  technical  feasibility  of  a  project.  But 
FFRDCs  can  also  become  embroiled  in  COIs; 
e.g.,  if  they  become  too  interested  in  growth,  or 
acquiesce  too  readily  to  improper  sponsor 
demands. 

5.  Freedom  to  concentrate  on  work 
assigned  by  their  sponsors  without  wasting 
resources  on  competing  for  new  work.  This 
recognizes  that  competition  is  not  free. 
Performers  who  compete  for  government  work 
typically  have  large,  expensive  marketing  and 
proposi  staffs. 

6.  Intimate  familiarity  with  sponsor 
activities  and  needs.  This  is  based  on  the 


536 


FFRDC’s  "special  relationship"  with  its  sponsor, 
allowing  quick  and  thorough  dispersion  of  plans 
and  technology  between  sponsor  and  FFRDC. 

7.  Broad  communication  of  R&D 
results.  FFRDC  mandates  to  disperse  new 
technology  throughout  the  industrial  community 
have  been  aided  by  recent  federal  legislation, 
including  the  National  Competitiveness 
Technology  Transfer  Act.” 

8.  Excellent  interdisciplinary 
capability.  FFRDCs  can  organize  around  and 
concentrate  on  well-defined  tasks  over  long 
periods  of  time.  Performers  who  are  constantly 
winning  and  losing  jobs  find  it  more  difficult  to 
maintain  interdisciplinary  teams. 

9.  Quick  response  to  the  sponsor’s 
needs.  This  is  due  to  the  FFRDC’s  close 
relationship  with  the  sponsor,  exemption  from 
competition,  and  freedom  from  federal 
regulations  that  slow  down  response  times. 

Disadvantages 

1.  High  costs  relative  to  other 
performers  because  of  their  nonprofit  status 
and  exemption  from  competition.  It  is 
difficult  to  compare  the  efficiency  of  FFRDCs 
with  commercial  performers  because  cost- 
effectiveness  measures  are  not  comparable  across 
programs.  One  study  conducted  by  individuals 
from  private  industry  found  no  significant 
difference  in  the  cost  structures  of  FFRDCs  and 
industry.  The  same  study  concluded  that  the 
fees  paid  to  DOD  FFRDCs  were  about  one-half 
those  paid  to  commercial  performers.* 

2.  Usurping  the  Government’s  policy 
functions  by  exercising  undue  influence  on  the 
policy-making  process.  This  criticism  is 
usually  leveled  by  elements  within  Congress, 
although  what  constitutes  "undue  influence" 
varies  from  person  to  person. 

3.  Adverse  effects  on  federal  in-house 
competence  due  to  FFRDCs  performing  the 
most  challenging  work.  There  may  be  merit  to 
this  view,  but  the  agency  sponsor  typically 
choo.ses  which  work  the  FFRDC  receives. 

4.  Absence  of  sufficient  di.scipline  to 
stimulate  high  levels  of  performance  due  to 
exemption  from  competition.  This  would  be 
hard  to  Ju.stify  given  the  numerous,  well 
publicized  successes  of  FFRDCs. 


5.  Unfair  advantage  in  obtaining 
R&D  work  due  to  exemption  from 
competition.  Commercial  performers  view  this 
as  a  valid  criticism.  But  from  the  viewpoint  of 
agency  sponsors  it  is  not  obvious  that  the 
exemption  has  been  a  bad  thing. 

6.  R&D  output  biased  toward 
sponsor’s  viewpoint.  Critics  of  FFRDCs  charge 
that  "he  who  pays  the  piper  calls  the  tune"  and 
that  FFRDCs  serve  the  interests  of  their 
sponsors  rather  than  the  public.  Many  also 
believe  that  profit-making  research  organizations 
have  less  reason  than  FFRDCs  to  be  biased.^' 

Another  disadvantage  aired  during  the 
late-1960s-early-1970s  debate  and  which  is  still 
claimed  today,  is  that  FFRDCs  have  out-lived 
their  usefulness.  Critics  point  out  that  FFRDCs 
were  created  in  the  crisis  atmosphere  of  World 
War  II  and  the  Cold  War  when  quick  reaction  to 
perceived  threats  was  essential.  It  was  prudent 
to  concentrate  resources  in  FFRDCs  to  focus 
energies  on  defense-related  science  and 
technology  tasks.  Now,  the  critics  say,  private 
industry  is  capable  of  performing  most  R&D 
tasks  performed  by  FFRDCs.  The  government, 
therefore,  should  acquire  more  R&D  from 
industry  in  keeping  with  its  traditional  reliance 
on  the  private  sector. 

ECONOMICS,  POLITICS,  AND  INSTITU¬ 
TIONAL  DESIGN 

Did  the  lack  of  public  debate  on 
FFRDCs  in  the  1940s  cause  the  criticisms  of  the 
late  1960s  and  early  1970s?  Certainly  many 
concerns  raised  by  the  critics  are  the  type  that 
would  have  been  addressed  during  an  earlier 
debate.  But  given  the  fundamental  economic 
and  political  implications  of  the  issues  raised,  it 
is  probably  more  accurate  to  say  that  the  lack  of 
earlier  debate  merely  delayed  the  surfacing  of 
the  criticisms.  Bruce  Smith  and  Joseph  Karlesky 
have  noted  that  the  competition  for  research 
funds  among  the  various  performers  became 
more  intense  as  federal  R&D  expenditures 
leveled  off  in  the  late  1960s.*  Such 
competition  in  all  likelihood  fueled  the  criticism 
of  FFRDCs.  Other  elements  of  the  debate  lead 
one  to  suspect  institutional  design  as  another 
explanation. 

Clearly,  there  is  substantial  disagreement 
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over  whether  certain  FFRDC  attributes  should 
be  considered  advantages  or  disadvantages, 
depending  on  one’s  institutional  partiality.  But 
institutional  attributes  are  a  product  of 
institutional  design.  The  institutional  design  of 
FFRDCs  is,  I  think,  a  major  cause  of  conflict 
with  commercial  performers  of  R«&D  and  with 
Congress.  This  can  be  better  understood  when 
one  recognizes  tha;  actors  in  the  science  and 
technology  policy  arena  operate  from  different 
paradigms  and  therefore  use  dif.'erent  conceptual 
strategies  to  Justify  government  action.  Harvey 
Averch’s  models  of  science  and  technology 
policy  Justification  are  useful  in  helping  to 
understand  these  differences.”  Two  models  in 
particular-the  engineering  and  market  models— 
provide  insight  into  how  proponents  of  different 
strategies  are  likely  to  view  institutional 
performers  of  R&D. 

Proponents  of  the  engineering  model 
view  the  government  as  having  primary 
responsibility  for  solving  major  social  and 
economic  problems.  Hence,  the  government  has 
an  interest  in  guiding  R&D  and  assuring 
appropriate  levels  of  funding  to  alleviate  such 
problems.  The  government,  having  greater 
access  than  the  market  to  information  about  the 
future  consequences  of  R&D,  must  actively 
promote  technology.  The  market  contains 
institutional  barriers  preventing  correct,  long¬ 
term  assessments  of  R&D.  Direct  action  by  the 
government  is  superior  to  indirect  action  because 
the  government  can  better  control  its  own 
activities  and  can  design  effective  programs.^ 

From  this  description  of  Averch’s 
engineering  model  it  is  clear  that  the  type  of 
institution  performing  R&D  should  matter  to 
engineering  advocates.  They  should  tend  to 
prefer  government  laboratories,  and  private 
industry  should  usually  be  their  last  choice. 
FFRDCs  should  fall  some  place  in  the  middle 
because  they  have  some  government  and  some 
private  attributes.  While  FFRDCs  are  tied 
closely  to  the  goals  and  programs  of  their 
sponsoring  agencies  and  do  not  have  the 
conflicts  of  interest  associated  with  private 
industry,  they  are  private  entities  and  they  do 
have  certain  freedoms  of  action  that  proponents 
of  the  engineering  model  would  find 
uncomfortable. 


In  the  late  1960s  and  early  1970s  as  well 
as  today,  proponents  of  the  engineering  model 
can  be  seen  in  Congress  and  the  bureaucracy. 
They  are  suspicious  of  FFRDCs  because  their 
institutional  design  enables  them  to  "circumvent" 
civil  service  and  procurement  regulations  and 
because  their  views  receive  great  deference  in 
agency  policy  making. 

But  FFRDCs  do  not  really  circumvent 
civil  service  rules;  the  rules  are  simply 
inapplicable  to  private  performers.  Instead 
FFRDCs  typically  pay  market  wages  and  salaries 
that  bear  a  strong  relationship  to  the  wages  and 
salaries  paid  by  their  parent  organizations.  As 
noted  above,  to  the  extent  FFRDCs  have 
exceeded  civil  service  pay  scales,  the 
government  has  benefitted  by  receiving  the 
services  of  scientists  and  engineers  it  could  not 
have  obtained  otherwise. 

In  the  procurement  arena  FFRDCs  are 
exempt  from  some  regulations  that  are  applicable 
to  other  contractors.  The  exceptions,  however, 
are  in  recognition  of  the  FFRDC’s  status  as  a 
full  member  of  the  sponsoring  agency’s  team, 
which  by  design  is  different  from  the  arms 
length  relationship  agencies  have  with  profit¬ 
making  contractors.  Also,  such  exceptions  allow 
FFRDCs  to  conduct  procurements  two  or  three 
times  faster  than  federal  agencies-an  ability  that 
is  often  essential  to  the  successful  completion  of 
agency  missions.  While  exceptions  to 
procurement  regulations  have  been  granted,  most 
FFRDCs  conduct  their  procurements  according 
to  the  federal  model,  including  the  requirement 
to  obtain  competition  when  practicable.” 

It  is  difficult  to  understand  the  concern- 
coming  primarily  from  members  of  Congress- 
over  the  influence  FFRDCs  have  in  the  policy¬ 
making  arena.  First,  agencies  are  free  to 
disregard  or  modify  FFRDC  advice  (which  they 
often  do).  Second,  agencies  receive  advice  from 
a  variety  of  outside  sources,  including 
universities,  commercial  performers,  members 
of  Congress,  and  various  special  interest  groups. 
Why  discriminate  against  FFRDCs,  which  may 
be  in  the  best  position  to  provide  objective 
advice  on  the  science  and  technology  policy 
issues  facing  their  agency  sponsors? 

Proponents  of  Averch’s  market  model 
believe  market  incentives  and  the  price  system 
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should  be  the  principal  means  of  delivering 
goods  and  services.  However,  society 
underinvests  in  R&D  because  the  market  for 
R&D  is  affected  by  high  risks  and  externalities. 
Govermnent  programs  to  correct  this  situation 
should  be  limited  primarily  to  support  for  basic 
research,  because  this  phase  of  R&D  suffers 
most  from  market  failure.  Direct  govenunent 
activity  in  the  later  phases  of  R&D  should  be 
limited  because  the  market  is  more  effective. 
Government  policies,  programs,  and  regulations 
are  barriers  to  efficient  R&D.“ 

Thus,  the  type  of  institution  performing 
R&D  should  also  matter  to  advocates  of  the 
market  model.  They  should  favor  private 
industry  as  the  primary  performer  of  R&D. 
They  should  also  view  government  performance 
of  R&D  as  an  improper  encroachment  on  the 
market.  As  with  proponents  of  the  engineering 
model,  market  advocates  should  have  mixed 
feelings  about  FFRDCs  but  for  the  opposite 
reasons.  They  should  be  uncomfortable  with  the 
close  ties  FFRDCs  have  to  the  goals  and 
programs  of  their  sponsoring  agencies  and  with 
their  exemption  from  competition.  On  the  other 
hand  proponents  of  the  market  model  should  be 
comforted  by  the  private  nature  of  FFRDCs  and 
the  freedom  of  action  that  could  make  them 
more  efficient  than  government  laboratories. 

In  fact,  private  industry  does  view 
FFRDCs  with  suspicion  because  their  institu¬ 
tional  design  exempts  them  from  competition 
and  other  principles  of  the  market  model.  In  the 
early  1970s,  the  National  Council  of 
Professional  Service  Firms  issued  a  statement 
calling  for  maximum  reliance  on  qualified  for- 
profit  performers  and  a  limited  role  for  DOD’s 
FFRDCs:  "The  fundamental  policy  which  should 
guide  support  of  captive  organizations  such  as 
FCRCs  (i.e.,  FFRDCs]  is  to  limit  their  activities 
to  those  for  which  the  private  sector  has  no 
competence  or  no  existing  capability.  Where 
there  exists  no  capability  in  the  private  sector 
and  the  government  needs  a  service  performed, 
this  service  may  be  performed  in-house  or 
through  an  FCRC  initially,  but  at  the  same  time, 
steps  should  be  taken  to  encourage  the  private 
sector  to  develop  such  capability  ...  at  the 
earliest  possible  time."’’ 

Many  in  industry  have  proposed  that 
FFRDCs  be  required  to  compete  for  their  share 


of  the  federal  R&D  dollar.  However,  as  the 
General  Accounting  Office  has  noted,  this  would 
alter  the  nature  of  an  FFRDC’s  special 
relationship  with  its  sponsor  by  subjecting  the 
FFRDC  to  the  uncertainties  of  the  market 
place.’*  If  FFRDCs  had  to  compete  they 
would  be  more  concerned  about  obtaining  new 
business  than  providing  unbiased  advice  to 
sponsors.  Because  FFRDCs  are  defined  in 
terms  of  such  a  relationship,  acceptance  of  this 
idea  would  be  tantamount  to  abolishing  FFRDCs 
as  an  alternative  way  of  meeting  government 
R&D  needs. 

NEW  RULES  AND  THE  CONTINUING 
QUEST  FOR  LEGITIMACY 

In  1972  the  U.S.  Commission  on 
Government  Procurement  (COGP)  issued  a  study 
report  on  the  federal  procurement  process  which 
recommended  that  the  government  keep  open  the 
option  of  using  FFRDCs  to  satisfy  "ne^s  that 
cannot  be  satisfied  effectively  by  other 
organizational  resources."”  In  response,  the 
Office  of  Federal  Procurement  Policy  (OFPP) 
issued  a  policy  statement  on  FFRDCs  that  is  still 
applicable  to  all  government  agencies.*  The 
statement,  which  superceded  the  1967  criteria 
promulgated  by  the  Federal  Council  for  Science 
and  Technology,  governs  the  establishment,  use, 
periodic  review,  and  termination  of  FFRDCs. 
It  provides  that,  in  establishing  FFRDCs,  the 
sponsoring  agency  must  ensure  that  (1)  existing 
sources  cannot  effectively  meet  the  special  R&D 
needs  of  the  agency,  (2)  public  notices  be 
published  indicating  the  agency’s  intention  to 
sponsor  an  FFRDC,  (3)  costs  of  FFRDC 
services  provided  to  the  government  are 
reasonable,  and  (4)  the  FFRDC’s  stated  mission 
clearly  indicates  the  kind  of  work  it  will 
perform. 

These  constraints  on  establishing 
FFRDCs  are  clearly  intended  to  quell  the  objec¬ 
tions  from  Congress  and  industry  that  were 
raised  in  the  late  1960s  and  early  1970s 
concerning  the  over-use  of  centers.  But  the 
policy  statement  goes  farther,  prohibiting 
FFRDCs  from  competing  with  any  non-FFRDC 
"in  response  to  a  federal  agency  formal  Request 
for  Proposal  for  other  than  the  operation  of  an 
FFRDC."*' 
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The  statement  has  given  FFRDCs  a 
degree  of  legitimacy  among  institutions  that 
perform  government  R&D.  It  recognizes  the 
unique  institutional  character  of  FFRDCs  by 
acknowledging  that  they  must  have  access 
"beyond  that  which  is  common  to  the  normal 
contractual  relationship,"^^  to  government  and 
supplier  data,  employees,  and  facilities  needed  to 
discharge  their  responsibilities,  even  if  that  data 
is  sensitive  or  proprietary.  The  statement  also 
recognizes  that  FFRDCs  should  not  be  subject  to 
the  same  market-oriented  rules  governing 
activities  of  profit-making  contractors.  It 
requires  sponsors  to  "undertake  the  respon¬ 
sibility  to  assure  a  reasonable  continuity  in  the 
level  of  support  to  the  activity  [i.e.,  FFRDC] 
consistent  with  the  agency’s  need  for  the  activity 
and  the  terms  of  the  sponsoring  agreement."" 

FUTURE  PROSPECTS 

In  the  last  few  years  the  number  of 
FFRDCs  has  again  increased.  The  current 
official  list  of  centers  from  the  NSF  includes 
41."  DOE  operates  22,  DOD  has  11  and  the 
remaining  8  are  spread  among  NASA,  NSF,  the 
Department  of  Health  and  Human  Services,  and 
the  Nuclear  Regulatory  Commission.  The 
increasing  numbers  of  FFRDCs  and  reduced 
defense  budgets  has  led  to  renewed  complaints 
from  Congress  and  industry  about  the  amount  of 
federal  funding  that  goes  to  centers.  Led  by  the 
Professional  Services  Council  (formerly  the 
National  Council  of  Professional  Service  Firms) 
and  the  U.S.  Chamber  of  Commerce,  industry 
has  again  pointed  to  the  exemption  of  FFRDCs 
from  competition. 

But  some  in  Congress  see  an  expanding 
role  for  FFRDCs.  Through  the  National 
Competitiveness  Technology  Transfer  Act  of 
1989,  FFRDCs  have  the  legislative  authority  to 
enter  into  cooperative  R&D  agreements  with 
private  industry."  DOE  centers  are 
particularly  interested  in  taking  advantage  of  this 
new  authority  in  light  of  massive  funding  cuts 
for  nuclear  weapons  research.  If  technology 
transfer  from  the  public  sector  to  the  private 
sector  is  successful,  FFRDCs  could  become  a 
force  in  improving  America’s  competitiveness  in 
international  markets. 


The  down-sizing  of  the  defense 
establishment  presents  other  possibilities.  In 
particular,  the  conversion  of  certain  R&D 
functions  performed  by  government  in-house 
laboratories  and  private  industry  to  the  FFRDC 
model  warrants  further  study.  The  Augustine 
Committee  briefly  discussed  the  conversion  of 
government  labs  to  FFRDC  status  in  their  report 
on  the  future  of  the  U.S.  space  program. 
Commenting  on  the  NASA/Caltech  arrangement 
for  the  management  of  the  Jet  Propulsion 
Laboratory,  the  Committee  stated  that  it  had 
"provided  an  enormously  effective  means  of 
obtaining  needed  technical  expertise  unfettered 
by  the  adverse  civil  service  restrictions.""  It 
recommended  that  NASA  selectively  phase 
additional  centers  into  the  JPL  model  by 
affiliating  them  with  universities  as  FFRDCs." 

The  conversion  of  segments  of  the 
defense  industry  to  FFRDC  status  could  allow 
for  the  orderly  reduction  of  excess  capacity  that 
currently  exists  at  the  major  systems  contractor 
level.  For  example,  segments  of  the  seven 
contractors  currently  capable  of  producing  air 
frames  might  be  ration^  ized  to  three  or  four 
FFRDCs-enough  to  eliminate  excess  capacity 
but  still  maintain  an  element  of  competition. 
This  would  not  mean  that  all  work  for  major 
aircraft  systems  would  be  performed  in-house  by 
FFRDCs.  Instead,  the  FFRDCs  would  perform 
as  R&D  centers  responsible  for  systems 
integration,  with  most  of  the  work  subcontracted 
to  private  industry.  This  would  require  an  arms 
length  relationship  between  the  FFRDC  and  its 
parent  and  might  require  the  parent  to  divest 
itself  of  certain  operations  to  avoid 
organizational  conflicts  of  interest. 

The  greatest  economies  would  probably 
occur  in  situations  where  government 
laboratories  and  private  enterprises  are  combined 
into  a  single  FFRDC.  This  would  allow  close 
coordination  among  the  processes  associated 
with  requirements  determination,  design, 
development,  and  systems  management.  Not 
only  would  redundancies  be  eliminated,  but 
time-consuming  and  costly  competitions  and 
contractual  disputes  would  be  greatly  reduced. 

This  proposal  would  not  do  away  with 
the  defense  industry  and  would  n^  amount  to 
nationalization.  It  would  convert  appropriate 
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parts  of  the  industry’s  top  layer-major  systems 
contractors-to  quasi  private  (or  public) 
FFRDCs.  The  remainder  of  the  industry  would 
remain  in  tact,  perhaps  supplemented  by 
operations  spun  off  by  companies  selected  to 
become  FFI^Cs. 

Others  will  argue  that  the  concept 
violates  the  government’s  long-standing  policy  of 
favoring  private  enterprise  in  the  procurement  of 
goods  and  services.  But  to  what  extent  is  the 
top  level  of  the  defense  industry  truly  "private 
enterprise"?  If  we  judge  it  on  the  basis  of  its 
adherence  to  free  market  principles,  it  falls 
short  on  the  "private"  side  as  well  as  the 
"enterprise"  side.  As  Don  Price  pointed  out 
almost  30  years  ago,  the  relationship  between 
the  government  and  business  for  the  planning 
and  conduct  of  R&D  programs  "is  more  like  the 
administrative  relationship  between  an  industrial 
corporation  and  its  subsidiary  than  the  traditional 
relationship  of  buyer  and  seller  in  a  free 
market."^  (This  already  sounds  a  lot  like  the 
relationship  between  the  government  and  an 
FFRDC.) 

Price  went  on  to  note  that  even  the 
contractual  relationship  "is  not  the  traditional 
market  affair."^  Contracts  are  not  awarded  on 
the  basis  of  competitive  bids;  sometimes  they 
are  not  competed  at  all.  The  contractor  does  not 
earn  a  profit  in  the  marketplace,  but  is  given  a 
fee  after  all  its  allowable  and  allocable  costs  are 
paid  for  by  the  government.  The  end  product 
cannot  be  specified  in  any  detail,  so  the 
processes  and  procedures  used  by  the  contractor 
to  achieve  an  end  product  are  specified  by  the 
government.  The  government  supplies  much  of 
the  plant  and  capital,  and  has  a  say  in  which 
costs  it  pays.  The  government  even  dictates 
how  contractors  account  for  costs. 

These  distortions  point  to  the  conclusion 
that  the  market  relationship  between  the  govern¬ 
ment  and  its  major  contractors  is  a  weak  one  at 
best.  Unlike  the  traditional  market  relationship 
in  which  there  are  many  buyers  and  sellers,  the 
marketplace  for  government  contractors  is 
characterized  by  one  buyer  (the  government)  and 
many  sellers  (the  contractors).  Thus,  the  normal 
give  and  take  that  occurs  in  a  free  market  does 
not  work  as  well. 


CONCLUSION 

The  first  step  in  helping  people 
understand  the  nature  and  proper  role  of 
FFRDCs  should  be  to  better  define  them.  The 
NSF  criteria  for  FFRDCs  are  inadequate.  At  a 
minimum,  criterion  S  should  be  revised  as 
follows:  "It  has  a  long-term  relationship  with  its 
sponsoring  agency  as  evidenced  by  (1)  relative 
stability  in  the  level  of  work  it  performs  for  the 
agency,  and  (2)  special  access  to  information 
concerning  agency  plans  and  programs."  In 
addition,  criteria  2,  3  and  6  should  be  eliminated 
as  they  add  nothing  meaningful  to  the 
definition.* 

The  top  level  of  the  defense  industry 
(which  to  some  extent  is  the  top  level  of  the 
space  and  nuclear  energy  industries)  is  closer  to 
the  FFRDC  model  than  many  would  like  to 
admit.  Converting  these  contractors,  and  certain 
government  labs,  officially  to  FFRDCs  would  no 
doubt  create  problems.  Issues  associated  with 
stockholder  equity,  intellectual  property  rights, 
and  organizational  conflicts  of  interest,  to  name 
a  few,  would  have  to  be  dealt  with.  From  a 
political  point  of  view,  opposition  would  come 
not  only  from  those  who  see  FFRDCs  as  too 
much  like  government  agencies,  but  from  those 
who  believe  they  are  too  much  like  private 
companies.  However,  now  is  the  time  to 
explore  this  possibility.  With  the  end  of  the 
Cold  War,  effective  rationalization  of  the 
defense  establishment  is  crucial  to  ensuring 
America’s  place  as  a  major  power  in  the  21st 
century.  Greater  use  of  the  FFRDC  model 
should  be  studied  as  part  of  the  rationalization 
process. 

Conflicts  over  the  proper  role  of 
FFRDCs  will  never  subside  completely.  But 
then  conflicts  over  the  proper  role  of  the  other 
institutional  performers  of  federal  R&D  have 
never  totally  subsided  either.  FFRDCs  are  a 
distinctive  and  useful  addition  to  the  array  of 
R&D  performers  available  to  the  government. 
As  long  as  government  is  obliged  to  focus  on 
day-to-day  operations  and  the  competing 
demands  of  our  pluralistic  society,  an 
incremental  approach  to  public  policy  is 
probably  inevitable.  Policy  makers  will  have 
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very  little  time  or  inclination  to  plan  creatively 
for  America’s  science  and  technology  future. 
The  independent,  interdisciplinary  nature  of 
FFRDCs  allows  for  the  development  and 
consideration  of  policy  alternatives  that 
otherwise  might  not  be  advocated  as  forcefully. 
This  paper  suggests  that  other  performers  may 
often  lack  the  capacity,  objectivity,  or  desire  to 
perform  this  function. 

FFRDCs  remain  an  effective  way  for 
government  to  attract  the  quantity  and  quality  of 
scientists  and  engineers  it  needs.  Their 
insulation  from  conflicts  of  interest  is  not 
perfect,  but  is  more  nearly  so  than  with 
commercial  performers.  In  short,  FFRDCs 
provide  government  with  an  institutional 
compromise  that  utilizes  the  efficiencies  and 
inducements  available  through  private  sector 
means,  while  assuring  a  high  degree  of  loyalty 
and  dedication  to  public  ends. 
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Abstract 

Recently,  there  has  been  renewed  inter¬ 
est  in  employing  modeling  and  simulation 
(M&S)  to  enhance  the  effectiveness  and  ef¬ 
ficiency  of  the  acquisition  process.  This  in¬ 
terest  has  been  manifested  in  recent  concepts 
such  as  synthetic  environments  and  virtual 
prototyping  which  are  designed  to  take  ad¬ 
vantage  of  the  burgeoning  technology  sup¬ 
porting  M&S. 

Over  the  past  thiny  years,  the  MITRE 
Corporation  has  made  extensive  use  of  M&S 
to  support  a  broad  spectrum  of  acquisition 
programs.  These  acquisitions  typically  in¬ 
volve  complex  information  systems  for  both 
the  military  community  and  civil  agencies. 
In  an  effort  to  distill  M&S  lessons  learned 
from  these  experiences,  a  number  of  in- 
depth  case  studies  have  been  performed.  By 
integrating  across  those  case  studies,  a  set  of 
key  issues  has  been  identified  that  are  per¬ 
ceived  as  being  central  to  the  effective  use 
of  M&S  in  the  acquisition  process.  These 
issues  involve  challenges  associated  with  the 
effective  creation  of  credible  M&S,  the  ef¬ 
fective  sharing  of  M&S  within  and  across 
community  lines,  the  development  of  effi¬ 
cient  methodologies  to  govern  the  use  of 
M&S,  and  the  injection  of  new  technology 
into  evolving  M&S.  For  selected  issues,  the 
results  of  the  case  studies  are  used  to  derive 
findings  on  the  use  of  M&S  in  the  acquisi¬ 
tion  process.  The  paper  concludes  by  for¬ 
mulating  a  set  of  recommendations  in  the  ar¬ 
eas  of  policy,  management,  standards,  and 
technology  that  should  serve  to  enhance  the 
effective  application  of  M&S  in  the  evolving 
acquisition  process. 


I.  Introduction 

Throughout  the  government  sector,  fun¬ 
damental  changes  are  underway  in  the  pro¬ 
cesses  by  which  complex  systems  are  ac¬ 
quired.  These  changes  are  being  driven  by 
the  confluence  of  several  factors:  resource 
constraints  (e.g.,  reductions  in  available  re¬ 
sources  due  to  the  burgeoning  national 
debt);  opportunities  offered  by  commercial 
products;  and  new  acquisition  paradigms 
(e.g.,  increased  emphasis  on  advanced  tech¬ 
nology  demonstrations).  These  factors  have 
stimulated  renewed  interest  in  using  M&S 
creatively  to  enhance  the  acquisition  of 
complex  systems. 

For  example,  DoD  has  recently  estab¬ 
lished  the  Defense  Modeling  and  Simulation 
Office  (DMSO)  to  help  stimulate  the  devel¬ 
opment  of  the  infrastructure  required  by  the 
users  of  M&S.  In  a  related  action,  the 
Director,  Defense  Research  &  Engineering 
(DDR&E)  has  initiated  a  set  of  Science  & 
Technology  thrusts  [1].  Prominent  among 
these  thrusts  is  an  effort  to  stimulate  the  de¬ 
velopment  of  “synthetic  environments.” 
These  environments  envision  the  possible 
interaction  of  a  variety  of  M&S,  including 
operations  with  real  equipment  in  the  field 
{live);  war  games,  models,  and  analytical 
tools  (constructive);  and  systems  and  troops 
in  simulators  fighting  on  simulated  battle¬ 
fields  (virtual). 

Similarly,  civilian  government  organizations 
are  seeking  enhanced  M&S  to  facilitate  the 
acquisition  of  complex  systems.  For 
example,  the  Federal  Aviation 
Administration  (FAA)  is  exploring  the  de 
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Figure  1  Selected  MITRE  Experiences 


Table  1  M&S  Projects  Considered  in  the  Case  Studies 


Type 

Acronym 

Project 

Communications  —  Systems 

JTIDS 

Joint  Tactical  Information  Distribution  System 

MIDS 

Multifunction  Information  Distribution  System 

MILSTAR 

Military  Satellite  Communications  System 

Communications  —  Networks 

FTS2000 

Federal  Telecommunications  System  2000 

JWICS 

Joint  Worldwide  IntelliKence  Communications  System 

MAGIC 

Multidimensional  Applications  Gigabet  Internet  Consortium 

Command  &  Control  Systems 

ATCCS 

Army  Tactical  Command  and  Control  System 

IVHS 

Intellisent  Vehicle/Highway  System 

mv/AA 

Integrated  Tactical  Waming/Attack  Assessment 

MCE 

Modular  Control  Element 

SPADOC 

Space  Defense  Operations  Center 

Information  System  Support 

APS 

Automated  Patent  System 

lAFIS 

Integrated  Automated  Fingerprint  Identification  System 

TSM 

Tax  Modernization  System 

Sensors/C^  Nodes 

AWACS 

Airborne  Warning  and  Control  System 

Joint  STARS 

Joint  Surveillance  &  Target  Attack  Radar  System 

IntelligeiKe 

Sentinel  Aspen  1, 11 

Sentinel  Bright  II 

Infrastructure  —  General 

OSI 

Open  System  Interface 

Space  Station 

Infrastructure  —  M&S 

AFOR 

Automated  Forces 

ALSP 

Aggregate  Level  Simulation  Protocol 

lUb 

Integration  and  Interoperability  Laboratory 

NASPAC 

National  Airspace  System  Performance  Analysis  Capability 

NSC 

National  Simulation  Center 

NTF 

National  Test  Facility 

Warbreaker 

Project  Model 

Project  FIRE 

Project  Facility  for  In-place  Realistic  Emulation 
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velopment  of  a  National  Simulation 
Capability  to  support  the  design  and 
development  of  the  next  generation  air 
traffic  control  system. 

This  paper  was  developed  to  share  the 
insights  that  MITRE,  a  Federally  Funded 
Research  and  Development  Center 
(FFRDC),  has  acquired  in  applying  M&S  to 
the  acquisition  of  complex  systems.  To  real¬ 
ize  that  goal,  four  inter-relaied  objectives 
have  been  identified.  First,  the  paper  high¬ 
lights  the  benefits  that  can  be  obtained  from 
the  appropriate  use  of  M&S  in  acquiring 
complex  systems.  Examples  are  cited  of  the 
benefits  that  have  been  derived.  Second,  the 
paper  identifies  and  discusses  critical  issue 
areas  that  practitioners  of  M&S  must  con¬ 
front.  For  selected  issue  areas,  significant 
findings  are  derived.  Finally,  the  paper  for¬ 
mulates  recommended  courses  of  action  for 
the  community  to  undertake  to  resolve 
residual  issues. 

This  paper  focuses  on  experiences  in 
applying  M&S  to  the  system  acquisition 
process.  For  the  purposes  of  this  paper,  sys¬ 
tem  acquisition  is  subdivided  into  five 
phases;  concept  formulation,  requirements 
definition,  development,  test  and  evaluation, 
and  operator  training.  Examples  will  be 
given  of  M&S  experiences  that  fall  within 
and  across  these  categories.  In  addition, 
consideration  will  be  given  to  operations  & 
maintenance  (O&M)  because  of  the  empha¬ 
sis  of  concurrent  engineering  in  system  de¬ 
velopment. 

The  bulk  of  the  acquisition  work  treated 
in  this  study  is  concerned  with  command 
and  control  systems,  both  for  military  and 
civilian  clients.  Accordingly,  the  paper  em¬ 
phasizes  the  lessons  learned  from  applying 
M&S  to  a  broad  spectrum  of  command  and 
control  systems.  However,  the  case  studies 
have  also  included  systems  to  deal  with  en¬ 
vironmental  issues  (e.g.,  chemical  demilita¬ 
rization),  space  (e.g.,  space  station),  and 
telecommunications  (e.g.,  the  Federal 
Telecommunications  System  2000). 
Consequently,  those  activities  have  been 
considered  in  the  identification  and  analysis 
of  M&S  issues. 


n.  Study  Approach 

This  study  is  based  upon  recent  experi¬ 
ences  in  support  of  systems  acquisitions  in 
both  the  military  and  civil  sectors.  The  is¬ 
sues,  findings,  and  recommendations  were 
developed  by  reviewing  the  results  of  M&S 
activities  across  the  company  and  integrating 
the  observations  from  individual  activities. 
This  included  reviewing  key  studies  and 
conducting  focus  group  discussions  to  elicit 
lessons  learned. 

These  experiences  in  M&S  support  of 
acquisition  can  be  related  to  specific  phases 
in  the  acquisition  cycle.  In  Figure  1,  selected 
projects  with  significant  M&S  content  have 
been  mapped  onto  the  concept  exploration, 
requirements  definition,  development,  test 
and  evaluation,  and  operator  training 
phases  of  the  acquisition  process.  In  general, 
M&S  work  in  support  of  these  projects  tran¬ 
scends  a  single  phase.  This  is  illustrated  by 
the  use  of  overlapping  regions  on  this  figure. 

The  projects  included  in  this  study  span 
a  wide  .*‘ange  of  types:  communications 
(systems  and  networks),  command  and  con¬ 
trol  systems,  information  system  support, 
sensors/C^  nodes,  intelligence,  and  infras¬ 
tructure  (general  and  M&S).  Table  1  maps 
the  projects  of  interest  into  these  categories. 
Technical  characterizations  of  these  projects 
can  be  found  in  [2]. 

III.  Major  Benefits 

The  general  benefits  of  M&S  are  out¬ 
lined  in  Table  2  for  each  phase  of  the  system 
life  cycle  from  concept  exploration  and  sys¬ 
tem  advocacy,  through  test  and  evaluation 
and  operator  training.  The  following  discus¬ 
sion  briefly  cites  examples  of  the  benefits 
that  have  been  derived  from  specific  pro¬ 
grams. 

In  support  of  system  advocacy  activities, 
M&S-based  analysis  can  provide  answers  to 
technical  questions  posed  by  decision  mak¬ 
ers.  In  the  case  of  the  Airborne  Warning  and 
Control  System  (AWACS),  M&S  was  used 
to  demonstrate  system  performance  in  the 
presence  of  jamming  and  screening. 
Similarly,  the  Joint  Surveillance  and  Target 
Attack  Radar  System  (Joint  STARS)  Radar 
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Table  2  Major  Benefits  of  M&S  in  the  Acquisition  Process 


Activity 

Berta  fits 

System  advocacy 

Concept  exploration 

•  New  Insights  Into  system  or  process  and  Hs  Integrated 
operation  In  Its  environment 

Requirements  formulation 

•  Clear  translation  of  users’  needs  to  a  well-defined  set 
of  technical  requirements 

•  Evaluation  of  coat,  schedule,  and  atfectlvanees 

Development 

•  Reduction  of  cost  and  risks  associated  with  design 
and  integration 

•  A  process  to  verify  design 

^  Efficient  arxl  cost-effective  planning  and  extension 
of  test  and  evaluation 

•  IdentlHcatlon/uryfarstandlng  of  critical  system 
parameters 

Operator  Training 

•  Reduced  lllacycia  cost  tor  training 

•  Better  trained  operators 

Evaluation  Activity  (REA)  model  was  used 
to  demonstrate  system  effectiveness  in  the 
presence  of  terrain  screening.  If  satisfactory 
answers  to  the  questions  had  not  been  found, 
it  is  likely  that  these  projects  would  have 
been  canceled. 

The  Joint  STARS  REA  model  was  also  very 
imponant  in  concept  exploration  activities. 
Joint  STARS  represented  a  new  mission  ex¬ 
ploiting  new  technology.  The  REA  was  also 
used  to  check  out  the  effectiveness  of  hu¬ 
man-system  interface  concepts  such  as  the 
use  of  compressed  history  displays  as  a  hu- 
man-in-the-loop  tracking  technique.  Due  in 
large  measure  to  the  contribution  of  M&S, 
two  developmental  models  of  Joint  STARS 
were  deployed  successfully  in  operation 
Desert  Storm,  six  years  before  they  were  to 
achieve  Initial  Operational  Capability.  In 
the  Modular  Control  Element  (MCE) 
Project,  M&S  was  used  to  prove  that  fre¬ 
quency-agile  filters  were  not  necessary  to 
prevent  interference  between  four  HAVE 
QUICK  radios  operating  in  the  same  shelter. 
An  unnecessary  $30M  procurement  was 
thereby  terminated. 

M&S  can  be  used  to  explore  alternative 
system  concepts  with  significant  impact  on 
system  cost.  For  example,  in  the  Seek  Igloo 
AN/FPS-117  radar  development,  M&S 
played  a  major  role  in  system  upgrades  de¬ 
signed  to  reduce  operations  and  maintenance 
costs.  The  original  system  performance 
specification  was  used,  but  new  operations 
concepts  were  explored.  Simulation  of  the 


required  frequency  spectrum  of  the  radar 
signals  to  meet  the  system  mission  demon¬ 
strated  the  feasibility  of  solid-state  compo¬ 
nents.  The  resultant  upgrade  does  not  require 
operators  on-site.  It  employs  a  highly  redun¬ 
dant  solid-state  design  and  remotes  the  radar 
data  to  a  Regional  Operational  Control 
Center  (ROCC).  The  resulting  operations 
and  maintenance  budget  has  l^en  reduced 
by  60%  (from  $120M  to  $49M  in  FY91).  In 
addition,  staffing  was  reduced  from  1650  to 
270.  The  return  on  investment  of  this  up¬ 
grade  is  22%  over  five  years. 

M&S  can  play  a  vital  role  in  the  devel¬ 
opment  of  consistent  requirements  that  can 
be  traced  back  to  the  system  operational 
concepts.  In  the  Multifunctional  Information 
Distribution  System  (MIDS)  development, 
VHDL  (VHSIC  Hardware  Description 
Language)  simulation  of  application  spe¬ 
cific  integrated  circuits  uncovered  specifica¬ 
tion  inconsistencies  that  would  have  re¬ 
quired  an  estimated  $2.6M  rework  .  Since 
the  MIDS  development  is  a  multinational  ef- 
fon,  it  is  very  likely  that  the  problems  would 
have  been  magnified  as  information  was 
transferred  from  country  to  country.  In  the 
area  of  chemical  demilitarization,  i.e.,  the 
destruction  of  chemical  weapons  stockpiles, 
M&S  was  used  to  evaluate  the  comparative 
performance  and  cost  of  demilitarization 
plant  alternatives,  analyze  the  throughput  of 
a  plant  so  that  the  number  of  facilities  could 
be  scoped,  and  evaluate  risk  and  cost  of  mu¬ 
nition  transport  alternatives. 
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During  system  development,  it  is  diffi¬ 
cult  to  monitor  design  convergence  toward 
the  overall  system  requirements.  M&S  can 
be  employed  effectively  to  ascertain  if  the 
design  activity  remains  on  track.  During  the 
Joint  STARS  development,  for  example, 
REA-based  analysis  identified  problems 
with  the  data  processor  capacity  that  the 
contractor  was  planning  to  employ.  As  a 
result  of  this  analysis,  two  successive 
changes  in  the  number  of  processors  and 
their  processing  capability  have  been  made. 

M&S  support  to  test  and  evaluation 
(T&E)  can  be  significant  in  planning  tests, 
interpreting  data,  training  operators  for  the 
tests,  and  actually  providing  part  of  the  test 
environment.  In  the  Joint  Tactical 
Information  Distribution  System  (JTIDS) 
project,  a  terminal  simulation  was  developed 
to  serve  as  a  real-time  monitor  of  network 
performance,  halting  tests  when  malfunc¬ 
tions  or  errors  occurred.  Simulations  repro¬ 
duced  test  deficiencies  to  identify  test  ex¬ 
tensions  to  improve  test  efficiency.  An  F-15 
manned  simulator  was  used  to  train  pilots 
for  actual  flight  tests  as  well  as  to  augment 
flight  test  results. 

The  use  of  M&S  to  support  operator 
training  over  and  above  that  required  for  test 
and  evaluation  has  been  demonstrated  in  a 
number  of  projects.  In  Sentinel  Aspen  I  and 
II,  and  Sentinel  Bright  II  intelligence  sys¬ 
tems,  model-based  training  resulted  in  better 
trained  operators  and  reduced  life  cycle 
costs  for  training.  Moreover,  model-based 
training  can  be  reconfigured  to  train  opera¬ 
tors  rapidly  for  system  upgrades.  Training 
modules  were  also  developed  for  MILSTAR 
military  satellite  communications  system 
operators  to  illustrate  the  impact  of  satellite 
orbits  on  earth  area  coverage. 

When  properly  employed,  M&S  can 
serve  as  a  tool  to  optimize  the  cost  of  devel¬ 
opment  and  ownership  of  a  system.  It  is  typ¬ 
ically  this  objective  that  drives  the  use  of 
M&S  in  support  of  system  acquisition,  train¬ 
ing,  operations,  and  maintenance.  Yet  there 
is  another  issue  associated  with  the  devel¬ 
opment  of  system  models.  Experience  indi¬ 
cates  that  the  development  and  maintenance 
of  complex  system-of-system  architectures 
must  be  supported  by  the  availability  of  an 


infrastructure  of  interoperating,  verified,  val¬ 
idated,  and  accredited  (VV&A'ed)  models. 
The  requirements  for  model  compatibility 
with  the  infrastructure  are  not  driven  by  the 
individual  system  developer,  but  must  be 
developed  and  enforced  by  the  organization 
responsible  for  the  infrastructure.  Thus,  the 
effective  use  of  M&S  to  support  acquisition 
must  be  viewed  not  only  in  the  context  of 
optimizing  the  development  and  cost  of 
ownership  of  the  specific  system,  but  also  in 
the  context  of  analysis  of  the  interoperability 
and  effectiveness  of  the  system  interacting 
with  other  systems  and  humans  when  ex¬ 
posed  to  various  environments  and  threats. 
The  issue  of  M&S  Infrastructure  addresses 
the  problem  of  how  best  to  develop  the  tools 
necessary  to  support  system-of-system  eval¬ 
uation. 

A  single  system  model  cannot  satisfy  all 
potential  uses  outlined  here.  It  is  necessary 
to  envision  a  group  of  related  models  each 
addressing  a  specific  problem  area.  The  dis¬ 
cussion  of  the  Evolution  of  M&S  throughout 
the  Acquisition  Cycle  addresses  the  relation¬ 
ship  of  models  used  at  different  stages  of  the 
acquisition  process. 

IV.  Key  Issues,  Findin  gs,  & 
Recommendations 

In  analyzing  the  individual  issues  that 
emerged  from  prior  experiences,  it  became 
apparent  that  they  could  be  embedded  into 
four  macro,  interrelated  issue  areas:  M&S 
creation,  M&S  use,  tools,  and  data.  The 
relationship  among  these  areas  is  illustrated 
in  Figure  2. 

In  the  area  of  M&S  creation,  four  issues 
were  identified:  the  technology  needed  to 
support  the  creation  of  M&S;  the  strategy 
that  should  be  used  in  developing  models; 
the  evolution  of  M&S  through  the  acquisi¬ 
tion  cycle;  and  the  VV&A  of  M&S.  In  the 
area  of  M&S  use,  three  issues  were  identi¬ 
fied:  the  community  sharing  of  M&S;  the 
Systematic  Use  of  M&S,  the  role  of  virtual 
prototyping  in  the  acquisition  process.  The 
strategy  that  should  be  used  in  developing 
the  M&S  infrastructure  to  meet  the  needs  of 
the  acquisition  community  emerged  as  the 
key  tools  issue.  Finally,  the  activities  re- 
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quired  to  satisfy  the  acquisition  communi¬ 
ty’s  data  needs  was  identified  as  the 
paramount  data  issue. 

In  this  paper  a  selection  of  the  issues  are 
discussed  in  detail.  These  include  Evolution 
of  M&S  Through  the  Acquisition  Cycle, 
VV&A,  Community  Sharing  of  M&S,  the 
Systematic  Use  of  M&S,  the  Role  of  Virtual 
Prototyping  in  the  Acquisition  Process, 
M&S  Tools,  and  Data  Needs.  The  summary 
includes  a  synopsis  of  the  findings  and  rec¬ 
ommendations  of  all  the  issues  considered  in 
this  study. 


A.  M&S  Creation 


Issue:  Evolution  of  M&S  Through  the 
Acquisition  Cycle  DoD  is  drafting  an  M&S 
policy  that  requires  the  use  of  models  and 
simu'ations  at  each  major  milestone  in  the 
acquisition  process.  The  relationship  of  the 
models  at  one  step  in  the  acquisition  process 
to  those  at  another  step  is  currently  not  de¬ 
fined.  On  the  basis  of  cost  considerations 
alone,  one  should  promote  model  reuse  and 
refinement  as  the  system  development  pro¬ 
cess  advances.  However,  experience  with 
the  use  of  M&S  throughout  the  system  life 
cycle  suggests  that  model  reuse  has  quantifi¬ 
able  benefits  only  at  certain  points  in  the 
system  life  cycle.  At  other  points,  indepen¬ 
dent  model  development  should  be  ex¬ 
ploited.  Moreover,  legal  restrictions  prohibit 
the  use  of  contractor  models  for  Operational 


M&S  CREATION 

•  Technology 

•  Development  slrategyj 

•  Evolution 

•  VV&A 


1 

•  Ini 


Figure  2 


Test  and  Evaluation  (OT&E). 

M&S  is  an  application-driven  endeavor. 
As  such,  the  model  developed  at  one  stage 
of  the  acquisition  process  may  have  been  de¬ 
signed  to  provide  answers  to  a  different 
range  of  questions  than  must  be  investigated 
at  another  stage  of  the  acquisition  process. 
Care  must  therefore  be  exercised  in  the  use 
of  models  developed  at  a  different  stage  of 
the  acquisition  process.  Alternatively,  con¬ 
sideration  must  be  given  during  model  de¬ 
velopment  to  the  possible  uses  of  the  model 
at  later  stages.  Modular  development  of  the 
model  to  provide  a  simple  means  of  enhanc¬ 
ing  the  model  capabilities  to  meet  later  re¬ 
quirements  should  be  pursued.  It  is  also 
mandatory  that  models  be  adequately 
VV&A’ed  prior  to  their  use  at  later  stages  in 
the  acquisition  process. 

It  has  been  found  that  the  development 
of  a  system  model  is  one  of  the  best  ways  to 
gain  insight  into  the  operation  and  require¬ 
ments  for  the  system.  It  is  therefore  recom¬ 
mended  that  government-developed  models 
not  be  used  directly  by  the  contractor  for 
system  development.  It  is  much  more  bene¬ 
ficial  for  the  contractor  to  develop  a  separate 
evolutionary  set  of  models.  The  models 
should  be  developed  by  a  closely-knit  team 
of  the  model  and  system  developers,  because 
when  the  coupling  between  the  models  and 
system  developers  is  not  adequately  clo.se, 
the  resulting  models  are  found  to  be  of  low 
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fidelity,  incorrect  or  misguided. 

Consequently,  one  can  envision  two  sets 
of  evolutionary  models:  a  government  set 
describing  system  requirements  and  envi¬ 
ronment,  and  a  contractor  set  dealing  with 
system  implementation.  The  two  model  sets 
should  be  correlated,  but  should  be  indepen¬ 
dent.  Moreover,  the  contractor  models 
should  be  deliverables.  A  number  of  bene¬ 
fits  result  from  requiring  the  contractor  to 
deliver  development  models  to  the  govern¬ 
ment.  These  include  better  insight  by  the 
government  into  how  well  the  design  is  con¬ 
verging  on  the  requirements,  better  evalua¬ 
tion  of  the  impact  of  engineering  change 
proposals,  and  more  accurate  forecasting  of 
how  the  system  performance  will  improve 
with  maturing  technology. 

Issue:  VV&A  There  is  increasing  aware¬ 
ness  in  the  M&S  community  of  the  need  to 
perform  VV&A.  However,  there  is  no 
agreement  on  the  paradigm  that  should  be 
applied  in  performing  those  functions.  As  a 
consequence,  there  is  little  appreciation  of 
the  resources  (e.g.,  cost,  time)  that  should  be 
allocated  to  the  VV&A  of  M&S,  as  a  func¬ 
tion  of  its  size  and  complexity. 

In  recent  years,  the  Military  Operations 
Research  Society  (MORS)  has  undertaken 
several  initiatives  on  the  subject  of  VV&A. 
In  the  late  1980’s,  it  established  a 
Simulation  Evaluation  Methodology 
Technical  Group  (SEMTG)  to  formulate  a 
high  level  concept  for  establishing  confi¬ 
dence  in  the  output  from  M&S  [3].  More 
recently,  MORS  has  convened  several 
Simulation  Validation  (SIMVAL)  work¬ 
shops  (4].  Through  those  workshops,  the 
community  is  converging  on  definitions  of 
the  terms  verification,  validation,  and  ac¬ 
creditation,  and  establishing  preliminary 
agreement  on  alternative  levels  of  VV&A 
(e.g.,  face,  constructive,  predictive). 

Based  upon  prior  experiences  with  the 
VV&A  process,  several  key  insights  have 
emerged: 

1.  It  has  become  apparent  that  VV&A 
must  be  begun  at  the  outset  of  a  program  and 
carried  out  throughout  the  entire  life  cycle. 
Efforts  that  defer  VV&A  to  the  end  of  a 
program  are  doomed  to  fail.  In  tact,  it  has 


been  found  that  “hooks”  must  be  included 
during  model  development  to  facilitate 
VV&A. 

2.  In  order  to  perform  VV&A,  it  is  nec¬ 
essary  to  clearly  articulate  the  assumptions 
that  have  been  made  in  creating  the  M&S.  It 
is  rare  that  creators  of  M&S  specify  these 
assumptions  clearly  and  completely. 

3.  The  final  step  in  the  process,  accredi¬ 
tation,  is  not  application  independent.  It 
must  be  performed  in  the  context  of  each 
specific  application  of  the  M&S. 

4.  For  the  limited  circumstances  under 
which  VV&A  is  currently  performed,  it  is 
generally  too  narrowly  scoped.  For  exam¬ 
ple,  in  virtual  simulations  (or  constructive 
simulations  in  which  controllers  are  integral 
elements),  the  experience,  training,  and  mo¬ 
tivation  of  the  humans-in-the-loop  are  essen¬ 
tial  elements  of  the  VV&A. 

5.  Many  current  attempts  at  accreditation 
tend  to  be  incomplete.  It  is  rare  that  they 
specify  the  degree  to  which  M&S  results  are 
applicable,  cr^ible  (e.g.,  should  the  user  of 
the  M&S  believe  the  product’s  trends,  rela¬ 
tive  output,  or  absolute  output?),  or  robust 
(e.g.,  credible  in  the  face  of  changing,  un¬ 
controllable  parameters). 

6.  The  community  has  not  yet  faced  up 
to  the  problem  of  VV&A  for  confederations 
of  interoperating  distributed  simulations.  It 
is  necessary,  but  not  sufficient,  that  simula¬ 
tions  be  valid  in  a  standalone  mode. 
Additional  steps  must  be  taken  to  establish 
the  validity  of  the  aggregate  set. 

7.  It  must  be  acknowledged  that  there 
may  not  always  be  a  “solution”  to  the 
VV&A  problem.  This  is  particularly  appar¬ 
ent  for  selected  hypothetical  future  systems 
for  which  an  incomplete  experimental  basis 
is  available.  Under  those  circumstances,  ex¬ 
treme  care  should  be  taken  in  interpreting 
and  employing  the  results  from  such  M&,S. 

In  view  of  the  criticality  of  this  subject, 
it  is  imperative  that  efforts  be  undertaken  to 
build  on  MORS’  experience  to  develop  and 
document  a  practical  approach  to  VV&A. 
That  effort  should  derive  rules  of  thumb  to 
suggest  the  resources  needed  to  perform 
VV&A,  undertake  case  studies  to  guide  im- 
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plementation  and  methcxiology,  and  seek  in¬ 
novative  ways  to  implement  VV&A  (e.g., 
use  visualization  tools  to  display  critical  pa¬ 
rameters,  equations,  and  decisions). 

B.  M&S  Use 

Issue:  Community  Sharing  of  M&S  In  the 
present  environment  of  declining  resources 
to  support  system  life  cycle  activities,  the 
sharing  of  M&S  across  community  lines 
(e.g.,  across  functions  and  across  Services) 
becomes  incTeasingly  significant.  There  is  a 
tendency  not  to  share  M&S  for  a  multitude 
of  reasons,  some  more  valid  than  others. 
Generally,  each  M&S  application  necessi¬ 
tates  modifications  to  a  model  since  model¬ 
ing  is  application  specific.  If  the  models 
have  been  inadequately  documented, 
VV&A'ed,  and  configuration  controlled,  the 
potential  users  of  a  legacy  model  have  little 
or  no  confidence  in  the  value  of  the  model 
and  tend  to  want  start  over.  In  addition, 
sharing  of  models  implies  loss  of  control  and 
closer  scrutiny  of  the  models.  Finally, 
models  are  often  not  shared  because  their 
existence  may  not  be  known  outside  of  the 
community  in  which  they  were  developed. 

Experience  suggests  that  the  reuse  of 
legacy  models  can  be  cost  effective  and  fa¬ 
cilitate  the  achievement  of  ambitious  M&S 
and  project  goals.  The  same  environments 
used  for  training  have  been  identified  as  op¬ 
portunities  for  concept  development  and 
other  acquisition-related  M&S  activities. 
Conversely,  acquisition  M&S  environments 
offer  opportunities  for  training  and  opera¬ 
tions.  Even  though  the  advantages  of  com¬ 
munity  sharing  of  M&S  have  been  demon¬ 
strated,  such  sharing  represents  the  excep¬ 
tion  rather  than  the  rule.  Poor  discipline  in 
model  development  and  documentation  im¬ 
pedes  sharing.  Moreover,  the  value  of  M&S 
sharing  is  dependent  upon  the  potential 
user's  domain  knowledge  (i.e.,  understand¬ 
ing  of  the  limitations  of  the  community 
model).  Cross  organizational  sharing  has 
been  stimulated  by  the  development  and  ap¬ 
plication  of  protocols  such  as  the  Aggregate 
Level  Simulation  Protocol  (ALSP)  and  sup¬ 
porting  software  that  permits  the  interoper¬ 
ability  of  disparate  M&S  |5). 


To  facilitate  community  sharing  of  M&S 
it  is  recommended  that  a  clearinghouse  of 
M&S  be  established  so  that  potential  users 
of  M&S  are  more  aware  of  community 
products.  Policies  to  prescribe  sound  soft¬ 
ware  engineering  principles  for  M&S  devel¬ 
opment  and  maintenance  should  be  formu¬ 
lated  and  implemented.  Cross-organiza¬ 
tional  and  cross-functional  interoperability 
of  disparate  M&S  should  be  enhanced.  The 
ALSP  experience  should  prove  useful  in  im¬ 
plementing  this  last  recommendation. 

Issue:  How  to  Use  M&S  Systematically  If 
M&S  are  to  be  cost  effective,  the  number  of 
simulation  runs  should  be  minimized.  At 
the  same  time,  however,  the  results  must  be 
statistically  significant  and  unbiased.  Since 
most  simulations  employ  some  element  of 
stochastic  modeling,  it  is  necessary  to  at¬ 
tribute  a  confidence  level  with  the  observed 
outcome.  Running  more  simulations  helps 
to  tighten  the  confidence  level  of  the  results. 
However,  repeating  human-in-the-loop  sim¬ 
ulation  runs  pose  some  special  problems. 
With  humans-in-the-loop,  there  is  a  need  to 
avoid  or  compensate  " ->1  effect  of  learn¬ 
ing  on  the  M&S  outt'  ne.  i  c  .rting  will  oc¬ 
cur  if  the  human  is  ,  i’rj  :d  to  what  is 
transparently  the  same  scenario  in  repeated 
simulation  runs. 

Frequently,  a  significant  portion  of  M&S 
resources  are  expended  in  VV&A  of  models 
and  scenarios  with  little  effort  devoted  to  the 
development  of  a  plan  to  exercise  the  resul¬ 
tant  M&S  systematically.  It  is  not  unusual 
for  M&S  sensitivity  studies  to  vary  a  single 
parameter  at  a  time,  even  though  design  of 
experiments  techniques  suggest  that  this  is 
neither  an  efficient  or  correct  approach. 

The  specific  experimental  design  se¬ 
lected  for  an  M&S  activity  is  dependent 
upon  the  nature  of  the  activity  and  the  con¬ 
straints  under  which  it  is  being  performed. 
Nevertheless,  formal  design  of  experiments 
techniques  provide  a  sound  foundation  to 
guide  the  exercise  of  the  M&S.  For  human- 
in-the-loop  M&S,  there  arc  blocking  designs 
that  minimize  learning  effects.  Adaptive 
strategies  are  generally  preferred  so  that  the 
M&S  activity  can  be  focused  on  the  most 
interesting  segments  of  the  solution  space. 
New  methodologies  have  been  introduced 
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recently  to  support  efficient  sensitivity  anal¬ 
ysis.  One  example  is  the  technique  pio¬ 
neered  by  G.  Taguchi  [6].  Taguchi  methods 
have  found  strong  support  in  U.S.  and 
Japanese  manufacturing  environments  and 
their  application  to  the  M&S  environment  is 
straightforward. 

The  use  of  sound  experimental  designs 
should  be  strongly  encouraged  by  suitable 
policies.  The  implementation  of  such  poli¬ 
cies  requires  guidelines  for  the  M&S  com¬ 
munity  on  how  to  develop  appropriate  ex¬ 
perimental  designs  and  how  to  conduct 
sensitivity  analyses.  Finally,  it  is  recom¬ 
mended  that  tool  development  to  support 
systematic  M&S  usage  should  be  encour¬ 
aged.  Examples  of  such  tools  include  meth¬ 
ods  of  configuring  models  for  specific  tests 
and  visualization  tools  to  illustrate  the  effect 
of  changing  parameters  on  simulation  re¬ 
sults. 

Issue:  The  Role  of  Virtual  Prototyping  in 
the  Acquisition  Process  A  prototype  may 
be  viewed  as  a  limited  functionality,  real¬ 
time  model  of  the  system.  Typically,  a 
prototype  is  used  to  explore  the  require¬ 
ments  for  the  user-system  interface  and  sig¬ 
nificant  effort  is  spent  in  providing  an  accu¬ 
rate  rendition  of  that  portion  of  the  system. 
There  is  a  general  consensus  as  to  the  impor¬ 
tance  and  utility  of  prototypes  in  helping  to 
define  the  user-system  interface.  In  addi¬ 
tion,  the  utility  of  prototypes  as  an  advocacy 
tool  is  generally  recognized. 

A  virtual  prototype  is  a  prototype  that 
can  interface  with  models  of  other  systems 
in  an  M&S  infrastructure  (synthetic  envi¬ 
ronment).  The  concept  of  a  virtual  proto¬ 
type  is  attractive  because  it  promises  to  re¬ 
duce  the  cost  of  system  development  by 
eliminating  costly  breadboard  and  brass- 
board  developments.  In  addition  it  has  the 
potential  of  involving  the  warrior  as  an  ac¬ 
tive  participant  in  the  system  acquisition 
process. 

The  virtual  prototype  concept  can  be 
useful  only  in  conjunction  with  a  synthetic 
environment  which  contains  an  established, 
accepted  M&S  infrastructure.  Therefore,  it 
is  necessary  to  develop  the  M&S  infrastruc¬ 
ture  to  make  virtual  prototyping  feasible. 


A  number  of  questions  remain  to  be  re¬ 
solved  concerning  the  interpretation  of  the 
results  of  virtual  prototyping:  How  does  one 
perform  VV&A  on  a  virtual  prototype? 
What  are  the  major  risks  associated  with 
basing  decisions  on  the  results  of  studies  that 
employ  virtual  prototypes?  The  virtual 
prototype's  environment  is  also  synthetic; 
therefore,  decisions  are  being  based  on  the 
use  of  limited  functionality  models  in  a 
synthetic  environment. 

It  is  therefore  recommended  that  the 
M&S  infrastructure  required  to  use  the  vir¬ 
tual  prototyping  concept  be  developed,  and 
specific  techniques  for  virtual  prototype 
VV&A  be  identified.  Virtual  prototypes 
should  be  used  within  the  context  of  a  for¬ 
mal  experimental  design  so  that  the  bounds 
on  sy.stem  performance  and  effectiveness 
can  be  estimated.  Finally,  the  Computer 
Aided  Design  community  should  be  stimu¬ 
lated  to  produce  tools  that  create  virtual 
prototypes  from  designers'  databases. 

C.  M&S  Tools 

Issue:  M&S  Infrastructure  An  M&S  in¬ 
frastructure  consists  of  the  model  set, 
databases,  and  networks  that  the  community 
assembles  as  a  context  for  performing  live, 
constructive,  or  virtual  simulations. 
Frequently,  it  can  cost  in  excess  of  $100M 
and  take  years  to  create  needed  infrastruc¬ 
ture.  This  poses  a  critical  question;  what 
strategy  should  be  used  in  developing  the 
M&S  infrastructure  in  order  to  meet  the 
needs  of  the  C^l  acquisition  community? 
Historically,  several  different  strategies  have 
been  used  by  developers  of  M&S  infrastruc¬ 
ture.  These  have  ranged  from  “build  it  and 
they  will  come”  to  the  building  of  a  well  fo¬ 
cused  set  of  narrowly  scoped  M&S  to  re¬ 
spond  to  specific,  well-defined  issues. 

Prior  experiences  have  revealed  prob¬ 
lems  with  the  strategies  associated  with  both 
extremes.  In  several  instances,  M&S  infras¬ 
tructure  has  been  built  and  users  have  not 
materialized.  This  failure  is  often  a  conse¬ 
quence  of  one  or  more  contributing  factors 
(c.g.,  the  “not  invented  here”  syndrome;  the 
costs  repaired  to  modify  the  infrastructure  to 
meet  the  user’s  unique  needs;  the  time  re¬ 
quired  for  the  potential  user  to  get  “up  to 
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speed”  on  the  infrastructure’s  capabilities 
and  uses;  limited  control  over  the  infrastruc¬ 
ture  on  the  part  of  the  individual  user). 
Alternatively,  the  characteristics  of  many 
C^l  issues  (e.g.,  complexity,  compressed 
schedule)  make  it  infeasible  to  create, 
VV&A,  and  exercise  a  M«feS  in  the  allotted 
time  if  no  infrastmeture  is  available. 

In  contrast  to  the  two  extreme  strategies 
cited  above,  an  evolutionary  acquisition  ap¬ 
proach  to  M&S  infrastructure  has  proven 
useful  (e.g.,  FAA  I-Lab).  In  this  approach, 
the  creator  of  the  M&S  infrastructure  ini¬ 
tially  develops  an  M&S  architecture,  em¬ 
phasizing  the  use  of  commercial  standards 
and  COTS  products.  Within  the  context  of 
that  architecture,  an  initial  core  capability  is 
created  rapidly  to  address  specific,  well-de¬ 
fined  issues.  Subsequently,  successive  in¬ 
frastructure  blocks  are  developed  that  reflect 
one  or  more  of  the  following  factors:  lessons 
learned  from  using  the  initial  increment(s)  of 
the  infrastructure;  capabilities  made  feasible 
through  M&S  technology  advances;  a 
clearer  understanding  of  the  user’s  needs.  In 
addition,  a  reliable,  accessible,  high-capacity 
backbone  network  is  an  essential  element  of 
M&S  infrastructure.  Even  with  growing 
network  capacities,  intelligent  data  distribu¬ 
tion  approaches  are  required. 

1).  Data 

Issue:  Data  Needs  Simulations  can  only  be 
as  credible  as  the  data  used  to  develop  and 
exercise  them.  It  is  unproductive  to  develop 
high-resolution  simulations  if  only 
approximate  data  is  available.  The  key 
question  that  the  M&S  community  faces  in 
ihc  area  of  data  is  the  acquisition,  mainte¬ 
nance  and  access  to  data  throughout  the 
M&S  life  cycle.  Credible  data  is  required 
during  M&S  creation,  VV&A,  and  during 
the  execution  of  M&S.  Examples  of  the 
data  needed  include:  force  structure,  combat 
service  support  (or  assets  and  support  for 
non  military  applications);  human-in-the- 
loop  performance;  tactics  and  operational 
prexedures;  and  environmental  factors  (e.g., 
terrain,  spectrum,  meteorology). 

Generally,  it  is  difficult  for  M&S  practi¬ 
tioners  to  find  out  about  the  existence  of 
credible  databases,  and  to  gain  access  to 


them.  In  addition,  databases  required  for 
M&S  are  large  and  should  be  supported  by 
flexible  database  management  systems. 

Significant  effort  is  required  to  acquire 
the  data  to  test  and  exercise  M&S.  When 
pre-scripted  scenarios  are  not  available,  their 
generation  is  both  time  and  resource  inten¬ 
sive.  Even  when  data  is  available,  it  is  gen¬ 
erally  necessary  to  transform  it  into  a  form 
appropriate  for  the  specific  application.  A 
thorough  understanding  of  the  significance 
of  each  data  item  is  necessary  to  insure 
proper  use  of  the  data  in  a  specific  M&S. 

Several  actions  are  recommended  to  im¬ 
prove  the  use  of  data  in  support  of  M&S  ac¬ 
tivities.  Standards  should  be  developed  to 
acquire  and  store  key  data  items  in  easily 
accessible  (within  the  constraints  of  security 
classification)  databases.  The  databases 
should  be  maintained  by  logical  experts  and 
used  on  a  read-only  basis.  Community¬ 
wide,  accepted  scenarios  should  be  gener¬ 
ated  for  common  use,  and  tools  should  be 
developed  to  facilitate  the  generation  of  new 
scenarios. 

V.  Summary 

A.  Major  Findings 

In  the  area  of  effective  creation  of  M&S, 
it  was  concluded  that  the  community  has  not 
adequately  addressed  several  critical  issues. 
First,  in  assessing  the  technology  that  pro¬ 
vides  the  underpinnings  for  M&S  it  was 
concluded  that,  with  few  exceptions,  ade¬ 
quate  technology  is  in  place  (or  in  progress) 
at  the  lower  technology  levels.  However,  at 
the  technology  application  level,  significant 
shortfalls  are  apparent,  particularly  in  human 
behavior  and  environmental  models,  instru¬ 
mentation,  multilevel  security,  and  auto¬ 
mated  forces.  Second,  under  the  auspices  of 
MORS,  the  subject  of  VV&A  has  recently 
come  under  increased  community  scrutiny. 
It  is  becoming  increasingly  accepted 
(although  infrequently  practiced)  that 
VV&A  must  be  begun  at  the  outset  of  a  pro¬ 
gram  and  carried  out  throughout  the  entire 
life  cycle  (in  the  context  of  each  specific 
application  of  the  M&S).  However,  there  is 
an  unsatisfied  need  for  paradigms  that  lead 
to  cost-effective  VV&A  procedures.  This  is 


554 


particularly  true  for  confederations  of  inter¬ 
operating,  distributed  simulations.  The  ap¬ 
propriate  strategy  for  M&S  development  is 
dependent  upon  the  specific  problem  being 
addressed.  Nevertheless,  it  is  best  to  start 
small  and  have  the  model  evolve  with  in¬ 
creasing  insight.  A  modular  architecture  can 
support  model  evolution.  Finally,  it  is  con¬ 
cluded  that  model  reuse  as  the  acquisition 
cycle  progresses  and  among  functional  areas 
and  services  is  feasible  only  with  a  well-de¬ 
fined  architectural  process  that  includes 
VV&A.  However,  there  are  a  dearth  of  the 
systems  architectures  and  engineering  prac¬ 
tices  that  are  needed  to  facilitate  reuse. 

In  the  area  of  effective  use  of  M&S, 
three  complementary  issues  have  been  iden¬ 
tified.  First,  it  has  been  demonstrated  that 
reuse  of  previously  developed  models  can  be 
cost  effective  and  facilitate  the  achievement 
of  ambitious  goals.  However,  poor  disci¬ 
pline  in  the  community  (e.g.,  poor  software 
engineering  practices,  inadequate  documen¬ 
tation  and  configuration  control)  impedes  ef¬ 
fective  community  sharing.  Consequently, 
the  sharing  of  M&S  across  community  lines 
represents  the  exception  rather  than  the  rule. 
Second,  a  formal  experimental  design 
(tailored  to  the  nature  of  the  M&S  and  con¬ 
straints  on  its  use)  provides  a  sound  founda¬ 
tion  to  guide  the  use  of  M&S.  Sensitivity 
studies  are  a  critical  dimension  of  this  use 
and  can  be  performed  effectively  using 
emerging  methods  (e.g.,  Taguchi  [6]  tech¬ 
niques).  Finally,  virtual  prototyping  can  be 
extremely  valuable  in  evaluating  system  ef¬ 
fectiveness  prior  to  system  build  and  in 
making  the  warrior  an  active  participant  in 
system  acquisition;  however,  it  can  only  be 
effective  in  conjunction  with  a  synthetic 
environment  containing  VV&A'ed  models, 
and  with  tools  that  support  prototype  devel¬ 
opment. 

In  the  area  of  tools,  the  development  of 
M&S  infrastructure  would  significantly  fa¬ 
cilitate  the  acquisition  of  complex  systems. 
However,  because  of  the  uncertain  nature  of 
the  requirements  for  most  of  these  systems 
(due  to  uncertainties  in  user  needs  and  tech¬ 
nology  developments),  it  is  desirable  to  ac¬ 
quire  the  M&S  infrastructure  in  an  evolu¬ 
tionary  manner. 


It  is  concluded  that  the  limited  availabil¬ 
ity  of  validated  data  adversely  effects  the 
creation  and  use  of  M&S.  In  general,  it  is 
difficult  to  find  out  about  the  existence  of 
credible  databases  or  to  gain  access  to  them. 
In  addition,  there  are  few  instances  where 
formal  mechanisms  have  been  established  to 
ensure  that  databases  are  current  (e.g.,  in¬ 
clude  latest  T&E  information)  or  validated. 

B.  Major  Recommendations 

In  the  area  of  M&S  creation,  significant 
enhancements  can  be  achieved  through  four 
key  initiatives. 

First,  there  is  a  need  to  establish  a  re¬ 
search  program  to  enhance  the  technology 
applications  that  are  the  foundation  for 
M&S.  The  greatest  needs  for  such  a  pro¬ 
gram  lie  in  the  areas  of  representing  human 
behavior  more  credibly  and  representing  a 
broad  spectrum  of  environmental  effects 
more  realistically  (e.g.,  terrain  effects  on  the 
dynamics  of  vehicles). 

Second,  creators  of  M&S  must  adopt  a 
well  articulated  development  strategy  at  the 
outset  of  a  program.  Experience  has 
demonstrated  that  it  is  folly  to  embark  upon 
a  M&S  development  without  a  clear  objec¬ 
tive  and  game  plan.  This  game  plan  should 
be  configured  to  anticipate  and  facilitate 
change.  While  this  recommendation  may 
appear  somewhat  obvious  at  first  glance, 
there  are  subtle  aspects  to  the  clear  articula¬ 
tion  of  the  objectives  of  the  M&S  activity. 

Consider,  for  example,  the  lack  of  uti¬ 
lization  of  the  Strategic  Defense  Initiative 
(SDI)  National  Test  Facility  (NTF).  An  at¬ 
tempt  was  made  to  create  a  general-purpose 
set  of  models  to  satisfy  a  wide  range  of  SDI- 
related  questions.  Unfortunately,  the  nature 
of  the  questions  and  the  required  timeliness 
of  the  answers  were  not  fully  factored  into 
M&S  development  activity  at  the  NTF.  By 
attempting  to  make  the  models  as  general- 
purpose  as  possible,  additional  development 
time  was  required.  The  M&S  development 
activity  lost  touch  with  the  overall  objectives 
of  the  study.  As  a  result,  the  models  were 
completed  after  the  required  answers  were 
needed  and  the  models  were  never  employed 
in  the  fashion  in  which  they  were  intended. 
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Third,  the  development  strategy  should 
evolve  models  during  the  acquisition  cycle 
along  two  separate,  coordinated  paths.  On 
the  government  side,  emphasis  should  be  on 
the  development  of  an  appropriate  system 
environment  to  facilitate  the  formulation  and 
refinement  of  requirements.  Conversely 
the  contractor  side,  the  focus  should  be  on 
the  development  of  tools  to  facilitate  system 
implementation.  Contracts  should  be  writ¬ 
ten  to  ensure  that  the  contractor  M&S  are 
fully  documented  deliverables. 

Finally,  in  recognition  of  the  criticality 
of  the  VV&A  process,  a  practical  approach 
to  the  process  must  be  developed  and  im¬ 
plemented  widely.  This  latter  effort  should 
build  on  recent  MORS’  experiences  in 
SIMVAL  and  SEMTG. 

There  are  several  steps  that  should  be 
taken  to  improve  the  use  of  M&S. 
Consistent  with  a  recent  Defense  Science 
Board  recommendation  [7],  efforts  should 
be  expanded  to  enhance  the  interoperability 
of  M&S  that  cuts  across  organizational  and 
functional  lines.  As  a  point  of  departure,  the 
community  should  build  upon  the  initial 
successes  achieved  through  the  ALSP. 
Second,  there  is  a  lack  of  understanding  in 
the  community  of  how  to  exercise  M&S  to 
derive  results  efficiently  that  are  statistically 
meaningful.  To  that  end,  it  is  vital  that  the 
use  of  sound  experimental  designs  be  en¬ 
couraged  to  govern  the  exercise  of  M&S. 
Finally,  the  trend  towards  virtual  prototyp¬ 
ing  is  to  be  applauded.  However,  if  that 
trend  is  to  be  meaningful,  it  will  require  the 
development  of  an  appropriate  M&S  infras¬ 
tructure  within  which  those  virtual  proto¬ 
types  can  be  embedded.  Moreover,  the  re¬ 
quirements  for  interfacing  virtual  prototypes 
to  the  M&S  infrastructure  must  be  clearly 
defined  and  disseminated  to  the  Computer 
Aided  Design  community.  Efforts  to  de¬ 
velop  the  capability  to  generate  virtual  pro¬ 
totypes  from  the  system  designers'  databases 
should  be  encouraged. 

In  the  area  of  M&S  infrastructure,  expe¬ 
riences  in  the  FAA’s  I-Lab  have  demon¬ 
strated  the  success  of  employing  an  evolu¬ 
tionary  approach.  In  this  approach,  the  in¬ 
frastructure  is  created  by  formulating  broad 


requirements  and  an  extensible  architecture. 
A  core  capability  is  created,  to  respond  to 
initial  needs,  and  evolved  in  blocks  to  reflect 
lessons  learned  from  its  use  and  to  take  ad¬ 
vantage  of  technological  advances. 

Finally,  in  the  area  of  community  data,  a 
major  initiative  is  required  to  respond  to 
widespread  needs.  This  includes  establish¬ 
ing  and  promulgating  standards  for  the  ac¬ 
quisition  of  data  and  establishing  and  main¬ 
taining  validated  databases  for  community 
use.  In  order  to  assure  the  integrity  of  the 
latter,  they  should  be  maintained  by  logical 
experts  and  used  on  a  read-only  basis. 
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Abstract 

Source  selection  practices  in 
government  differ  from  those  of  private 
sector  purchasers.  Reasons  for  the 
differences  are  attributable  to 
government  effort  to  introduce  social 
policy  through  purchases,  to  maintain 
fairness  in  the  award  of  government 
contracts,  and  unwillingness  to  rely  on 
judgment  of  its  administrators.  Detailed 
procedures  in  statute  and  regulation  and 
comprehensive  review  and  audit 
organizations  define  the  government 
methods.  However,  important  evidence 
is  being  developed  through  research 
that  the  private  sector  practices  result  in 
more  effective  buying  operations,  more 
competitive  markets,  and  a  higher  level 
of  satisfaction  in  the  buying  community. 
Suggested  modifications  of  government 
policies  are  based  on  a  review  of  the 
selected  research  reports  and  case 
decisions. 

Source  Qualification  and  Selection, 
Private  Sector 

For  approximately  ten  years,  private 
sector  purchasers  in  the  United  States 
have  been  revising  their  source 
qualification  and  selection  practices  with 
the  effect  of  reducing  the  size  of  their 
supplier  bases.  In  doing  so,  they  are 
developing  close  working  relationships 
with  their  remaining  suppliers  and 
demanding  order-of-magnitude 
improvements  in  reliability,  conformity 
with  requirements,  collaboration  in 


matters  of  design  and  market,  and 
service  that  is  fully  coordinated  with  the 
purchaser’s  needs.  To  achieve  these 
objectives,  they  are  adopting  new 
practices,  including  Early  Supplier 
Involvement  (ESI),  Electronic  Data 
Interchange  (EDI),  and  scheduled 
delivery  times.  Consequently,  there  has 
been  a  reduction  in  the  number  of 
competitive  purchases,  an  increase  in 
the  length  of  contractual  relationships 
and  increased  interdependence  of  buyer 
and  seller.  The  motivation  for  these 
changes  is  reduced  cost  and  increased 
competitiveness  for  the  buying 
company.  Experience  indicates  that  the 
reduced  level  of  competition  in 
purchases  results  in  increased 
competitive  power  for  the  purchasing 
company.  This  result  is  vital  to 
companies  that  aspire  to  compete  in  the 
emerging  global  marketplace. 

Government  Policy  Changes 

Coincident  with  these  developments 
in  the  private  sector,  the  United  States 
government  has  introduced  a  series  of 
modifications  in  its  purchasing  policies 
and  practices.  However,  governmental 
innovations  have  been  aimed  in  the 
reverse  direction.  The  Congress 
keynoted  this  period  by  adopting  the 
Competition  in  Contracting  Act  (CICA)  in 
1984,^  which  introduced  or  expanded 
many  policies  and  practices  focused  on 
mandating  increased  competition.  The 
central  policy  adopted  required  full  and 
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open  competition  for  all  government 
purchases  with  narrowly  defined 
exceptions.  in  1988  the  Congress 
adopted  the  Procurement  Integrity  Act 
with  the  explicit  intention  of  reducing 
interaction  between  government 
procurement  officials  and  contractor 
personnel.^  These  two  statutes  are  of 
particular  importance  in  the  continuing 
refinement  of  government  procurement 
processes  (other  enactments  have  been 
adopted  each  year,  but  most  focus  on 
narrower  issues  and,  overall,  probably 
are  less  significant).^  These  enactments 
have  caused  the  rules  to  be  restated  to 
emphasize  the  governmental  preference 
for  ad  hoc,  arms-length  purchasing, 
based  on  objective  source  selection 
criteria,  comprehensive  documentation, 
and  with  primacy  given  to  social  and 
fairness  objectives  over  program  end 
results. 

Recent  Research  in  Private  Sector 
Purchasing  Practices 

The  purpose  of  this  study  is  to 
highlight  private  sector  purchasing 
practices  by  summarizing  several  recent 
study  reports  in  which  currently 
developing  attitudes  and  techniques  are 
examined  through  empirical  and 
experimental  research.  After 
summarizing  several  such  studies,  a 
comparison  with  governmental  practice 
is  made  through  a  review  of  federal 
acquisition  rules  and  case  decisions  in 
which  the  federal  buying  practices  are 
subjected  to  analysis  by  the  General 
Accounting  Office. 

Research  giving  insight  into  private 
sector  purchasing  practices  indicates 
that  the  purchasers  are  adopting  a 
practice  of  purchasing  on  the  basis  of 
quality-specifically,  quality  that  is  built-in 
using  verified  in-process  quality 
assurance  procedures.  Interest  in  these 


developments  on  the  part  of  public 
buyers  is  indicated  by  their 
experimentation  with  "best  value" 
source  selection  procedures,  their  effort 
to  develop  techniques  for  evaluating 
past  performance  of  potential  suppliers, 
and  by  public  agencies’  attempts  to 
adopt  a  Total  Quality  Management 
philosophy. 

Industrial  buyers  use  a  variety  of 
practices  that  seem  to  deny  any  single- 
minded  dedication  to  price  competition.'* 
Plank  and  Kijewski^  reported  in  1991  on 
the  extensive  use  of  approved  supplier 
lists  by  a  sample  of  purchasing 
managers  drawn  from  thirty-two  different 
SIC  codes.  They  found  that  more  than 
70  percent  of  the  companies  surveyed 
establish  approved  suppliers  lists  for 
use  of  their  buyers.  Where  such  lists 
are  used,  it  seems  evident  that  selection 
of  suppliers  is  a  two-phase  process,  the 
first  phase  based  primarily  on  an 
assessment  of  qualitative  matters 
through  one-on-one  negotiation,  plant 
visits,  and  other  techniques.  The 
second  phase  may  or  may  not  impose 
a  price  competition  but  restricts  the 
competition  to  the  pre-qualified  buyers. 
These  lists  predominate  in  the  purchase 
of  production  materials  and  of  supplies 
and  services  but  are  used  to  a  lesser 
degree  for  capital  items.  Plank  and 
Kijewski  found  that  use  of  the  lists 
increased  as  the  complexity  of  the 
product  being  procured  increased. 
Their  study  was  of  private  sector 
buyers.  For  comparison,  it  should  be 
noted  that  severe  restrictions  on  this 
buying  practice  are  present  in  U.  S. 
government  procurement  as  is  evident 
from  the  restrictive  requirements  for 
establishing  qualified  product  lists  found 
in  the  Federal  Acquisition  Regulation 
(FAR)  in  Subpart  9.2.® 

Billesbach,  et  al,^  in  their  1986  study 
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of  JIT-oriented  companies  in  the  United 
States  and  Great  Britain  found  that 
purchasing  practices  in  their  sample  of 
manufacturers  appear  to  emphasize  a 
different  set  of  concerns  from  those 
associated  with  traditional  (non-JIT) 
manufacturers.  These  researchers 
found  heavy  emphasis  on  materia! 
quality,  reliable  deliveries,  delivery 
flexibility,  and  supplier  base  reduction; 
they  found  relatively  less  emphasis  on 
price  competition.  Their  study  required 
respondents  to  specify  the  relative 
priority  they  placed  on  supplier 
capabilities  and  programs.  When  their 
data  for  their  sample  companies  is 
combined,  it  reveals  that  purchasing 
managers  place  priority  on  supplier 
capabilities  and  practices  in  the 
following  proportions: 

Delivery  reliability  96  percent 

Conformance  to  quality  91  percent 

Response  to  changing  demands 
(flexibility)  69  percent 

Use  of  long-term  delivery 

schedules  68  percent 

Price  competitiveness  67  percent 

Adoption  of  fixed,  short-term 
scheduling  for  JIT 

deliveries  57  percent 

Adoption  of  zero  (incoming) 

inspection  programs  30  percent 

This  data  attributes  a  lower  level  of 

importance  to  price-based 
competitiveness  than  to  several  other 
factors.  It  does  not  allow  firm 
conclusions  on  the  relative  importance 
of  qualitative  matters  and  price,  but 
provides  support  for  the  notion  that  non¬ 
price  factors  may  predominate  in 
importance  for  JIT  purchasers.  Other 
measurements  of  supplier  performance 
examined  in  the  Billesbach  study 
included  suppliers’  technical  support 


and  JIT  capability.  The  study  indicated 
that  buyers  place  a  moderate  to  high 
priority  on  these  factors  (50  and  42 
percent  respectively  for  the  combined 
data  base).  Respondents  to  the 
Billesbach  survey  reported  that  they  had 
substantially  decreased  the  number  of 
their  suppliers.  Some  of  the 
respondents  (10  to  11  percent)  had 
achieved  greater  than  50  percent 
reductions  in  their  supplier  base  during 
a  five-year  period.  However,  their  data 
did  not  indicate  much  interest  in 
electronic  data  interchange,  sharing  of 
inventory  information,  deliveries  to  point 
of  use,  or  deliveries  within  specific  daily 
time  slots. 

Perry,®  in  1992,  described  an 
analytical  model  by  which  purchasers 
can  quantify  specific  qualitative  factors 
relevant  to  the  purchase  of  computer 
integrated  manufacturing  (CIM)  systems 
(the  technique  might  be  applied  to 
purchase  of  any  capital  equipment  item). 
The  model  poses  a  mechanism  whereby 
reliability,  performance,  durability  and 
serviceability  can  be  assigned 
quantitative  values  reflecting  the 
minimum  required  level  of  performance 
for  each.  These  values  are  then 
combined  into  a  single  measure- 
operating  effectiveness.  The 
purchaser’s  required  level  for  each  of 
the  parameters  (reliability,  performance, 
durability  and  serviceability)  may  also  be 
evaluated  to  assess  what  percentage 
premium  the  buyer  would  pay  for  a 
specific  percentage  improvement  in  the 
level  of  performance  beyond  that 
minimally  required.  In  a  similar  manner, 
the  operating  effectiveness 
measurement  may  be  evaluated  to 
determine  what  premium  might  be  paid 
to  secure  a  higher  level.  Using  this 
model  in  source  selection  would  enable 
the  decision  maker  to  consider,  on  an 
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objective  basis,  the  cost  and 
performance  capabilities  of  each  offeror 
and  would  enable  the  purchaser  to 
determine  the  optimum  cost-benefit  ratio 
for  selection  of  the  source.  This 
approach  requires  some  assumptions 
and  may  be  difficult  to  apply.  The 
purchaser  must  specify  a  minimum  level 
of  operating  effectiveness  as  well  as 
incremental  levels  of  effectiveness.  It 
also  requires  reliable  seller 
representations  regarding  capabilities  of 
proposed  systems.  The  purchaser  also 
must  be  capable  of  accurately  assigning 
values  to  the  incremental  levels  of 
capability.  Perry’s  model  represents  an 
effective  method  of  providing  objective 
criteria  for  dealing  with  specific 
qualitative  issues  in  source  selection. 
For  this  reason,  it  could  prove  useful  in 
the  private  sector  and  may  have  special 
appeal  to  government  buyers.  The 
model,  however,  does  not  resolve  the 
policy-related,  mainstream  issues  facing 
government  buyers,  specifically, 
adoption  of  long-term,  cooperative 
buyer-seller  relationships. 

The  concept  of  purchasing 
partnerships  has  captured  attention  in 
the  private  sector  buying  community 
during  the  last  five  years.  In  1991 
Ellram^  developed  a  normative  model  for 
creating  buyer-seller  partnership 
relationships.  Her  five-phase  model, 
which  assumes  an  affirmative  effort  by 
the  purchasing  company  to  form  the 
relationship,  was  derived  from  her  case 
studies  and  literature  published  from 
1986  through  1990.  In  addition  to  her 
model,  she  identifies  factors  likely  to  be 
correlated  with  successful  partnering. 
These  factors  include  total  cost  (not 
price)  as  an  objective  of  the  partnership, 
mutual  trust  and  openness,  reduction  in 
the  number  of  suppliers,  communication 
occurring  at  many  levels  and  functions 


between  the  firms,  and  sharing 
information  with  suppliers  regarding 
changes  and  new  product  designs  early 
in  the  process. 

Dumond^°  reported  an  experimental 
study  in  1991.  She  assessed  the 
measurement  of  purchasing  perform¬ 
ance  under  four  distinct  management 
systems  (each  system  communicated 
management’s  objectives  on  the  basis 
of  which  the  corporate  reward  structure 
was  built).  The  design  of  her  study 
required  240  subjects  (grouped  so  that 
one-fourth  of  the  total  were  guided  by 
one  of  the  four  sets  of  management 
criteria)  to  decide  a  course  of  action 
from  defined  alternatives  for  each  of 
fourteen  purchasing  problem  situations. 
Of  the  fourteen  situations,  twelve 
involved  purchasing  cost  savings, 
reductions,  or  avoidance  as  an  implicit 
alternative.  In  each  of  these  cases,  the 
alternatives  forced  the  cost  savings 
objective  to  be  traded  off  against  other 
benefits,  such  as  customer  satisfaction, 
quality,  delivery,  or  production  cost 
savings.  Upon  making  a  decision,  the 
subject  received  feedback  on  how  he  or 
she  was  performing  against 
management’s  criteria,  then  proceeded 
to  another  problem  to  decide  how  it 
should  be  treated.  One  of  the  four 
measurement  systems  emphasized 
efficiency,  i.e.,  purchase  cost  reduction, 
operating  cost  minimization,  and  order 
processing  time  reduction.  Another 
emphasized  effectiveness,  i.e.,  overall 
contribution  to  profit,  supplier  relations, 
and  customer  satisfaction.  The  third 
system  mixed  the  efficiency  and 
effectiveness  objectives  (multiple 
objective  system),  and  the  fourth 
provided  no  management  direction 
(naive  system).  Dumond  found  that 
when  performance  evaluation  was 
based  on  effectiveness,  the  subject 


560 


achieved  the  highest  contribution  to 
profit,  the  best  relationships  with 
suppliers,  and  the  greatest  satisfaction 
in  the  end-use  customer.  The  efficiency 
system  resulted  in  the  greatest  cost 
reduction  and  least  operating  cost.  In 
addition,  she  used  a  post-experience 
questionnaire  to  measure  the  240 
subjects’  perception  of  their  experience. 
She  found  that  the  group  guided  by  the 
effectiveness  system  generated  a  higher 
level  of  commitment,  a  clearer 
understanding  of  objectives,  a  higher 
feeling  of  confidence,  and  greater 
enjoyment  than  any  of  the  other  groups. 
It  is  evident  from  these  results  that  the 
effectiveness  system  guided  the 
respondents  to  achieve  the  greatest 
overall  benefit. 

Dumond’s  findings  are  an  important 
contribution  to  the  study  of  purchasing 
practices  because  they  show  clearly 
that  strict  orientation  of  purchasers  to 
price  and  cost  minimization  does  not 
accomplish  organizational  objectives 
nearly  as  well  as  a  broader  orientation. 
This  message  is  consistent  with  the 
growing  belief  that  industry  becomes 
more  competitive  under  partnership 
relationships  because  of  the  focus  on 
increased  communications,  delivery 
reliability,  quality  improvement,  and 
value  enhancement  to  the  customer  and 
the  supplier. 

Each  of  the  techniques  and  ideas 
summarized  from  the  cited  reports 
appears  to  have  merit  as  an  aid  to 
improving  productivity  in  the  United 
States,  and  some  of  the  ideas  could 
benefit  government  procurement.  All  of 
the  ideas  could  be  used  in  a  public 
procurement  process  if  the  policy  of 
governmental  authority  were  to  allow 
their  adoption,  but  such  policies  require 
increased  reliance  on  discretion  and 
judgment  by  administrative  personnel  in 


the  executive  branch.  The  trend  in 
government  is  in  the  reverse  direction: 
decreased  reliance  on  discretion  and 
judgment  within  the  bureaucracy. 

Government  Policies  and  Enforcement 

One  factor  in  the  resistance  of 
government  to  adoption  of  qualitative 
bases  for  source  selection  is 
accumulation  of  an  increasingly  detailed 
procurement  regulatory  scheme  and  the 
expansion  of  review  and  audit 
organizations  chartered  to  ensure 
compliance  with  the  rules.  These 
organizations,  which  include  the  DCAA, 
GAO,  Inspectors  General  and  other 
investigative  offices,  are  responsible  for 
supporting  the  contracting  process 
(DCAA);  for  ensuring  proper  use  of 
public  funds  (GAO);  and  for  ferreting  out 
waste,  fraud  and  abuse  associated  with 
public  expenditures  (Inspectors 
General).  These  are  vital  functions 
performed  in  the  public  interest. 
However,  these  organizations  have  no 
responsibility  for  accomplishing  end 
results.  A  few  examples  of  GAO 
decisions  in  award  protest  cases 
decided  during  the  period  April  1,  1991, 
through  July  1 , 1992,  are  examined  here 
as  illustrations  of  the  manner  in  which 
procurement  decisions  are  scrutinized.” 

GAO  protest  decisions  are  the  result 
of  review  by  the  GAO  legal  staff  of  the 
facts  and  circumstances  of  an  agency’s 
source  selection  decision  in  light  of  a 
careful  comparison  with  the  FAR  and 
other  regulatory  or  statutory  rule.  The 
FAR  has  been  growing  in  the  level  of 
detail  by  which  it  defines  the  procedures 
required  of  the  government’s  contracting 
officers.  To  illustrate  this  level  of  detail, 
selected  items  are  quoted  here.  Each 
of  the  quotes  has  been  applied  by  GAO 
in  recent  rulings  as  part  of  the  basis  for 
overturning  a  contracting  officer’s 
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source  selection  decision.  The 
quotations  are  from  FAR  Part  1 5  which 
deals  with  negotiated  procurement  and 
is  often  central  to  protest  decisions. 

FAR  15.406-5(c),  concerning  the 
preparation  of  solicitations  and  the 
inclusion  of  evaluation  factors  for  award, 
requires  the  contracting  officer  to 
"Identify  all  factors,  including  price  or 
cost,  and  any  significant  subfactors  that 
will  be  considered  In  awarding  the 
contract  ...  and  state  the  relative 
importance  the  Government  places  on 
those  evaluation  factors  and 
subfactors."  FAR  15.600,  which  states 
rules  for  selection  of  source,  contains 
detail  at  FAR  15.605  regarding  use  of 
the  evaluation  factors.  Paragraph  (e)  in 
that  subsection  states,  "The  solicitation 
shall  clearly  state  the  evaluation  factors, 
including  price  or  cost  and  any 
significant  subfactors,  that  will  be 
considered  in  making  the  source 
selection  and  their  relative  importance. 
Numerical  weights,  which  may  be 
employed  in  the  evaluation  of  proposals, 
need  not  be  disclosed  in  solicitations. 
The  solicitation  shall  inform  offerors  of 
minimum  requirements  that  apply  to 
particular  evaluation  factors  and 
significant  subfactors." 

At  FAR  15.608  evaluation  of 
proposals  is  constrained  by  the 
following:  "An  agency  shall  evaluate 
competitive  proposals  solely  on  the 
factors  specified  in  the  solicitation." 
Also  in  FAR  15.608(1)  "...  The 
contracting  officer  shall  document  the 
cost  or  price  evaluation,"  and  in  FAR 
15.608(2)  "...  the  cognizant  technical 
official,  in  documenting  the  technical 
evaluation,  shall  include-- 

(i)  The  basis  for  evaluation; 

(ii)  An  analysis  of  the  technically 
acceptable  and  unacceptable 
proposals,  including  an  assessment 


of  each  offeror’s  ability  to  accomplish 
the  technical  requirements: 

(iii)  A  summary,  matrix,  or 
quantitative  ranking  of  each  technical 
proposal  in  relation  to  the  best  rating 
possible;  and 

(iv)  A  summary  of  findings. 

Finally,  supporting  documentation  is 

addressed  at  FAR  15.612(d)(2)  which 
requires  that  "The  supporting 
documentation  prepared  for  the 
selection  decision  shall  show  the 
relative  differences  among  proposals 
and  their  strengths,  weaknesses,  and 
risks  in  terms  of  the  evaluation  factors. 
The  supporting  documentation  shall 
include  the  basis  and  reason  for  the 
decision." 

Each  of  these  quotations  is  a  brief 
extract  from  the  comprehensive 
treatment  of  this  subject  in  the  FAR. 
Nevertheless,  the  cited  items  are  lengthy 
and  precise  rules,  and  each  is  the 
product  of  the  regulation  writer’s  in- 
depth  analysis  of  what  might  be 
characterized  as  a  rational  set  of 
decision-making  rules.  Each 
requirement,  standing  alone,  makes 
sense.  In  its  protest  operations,  the 
GAO  uses  these  and  many  other  FAR 
rules  as  the  basis  for  deciding  whether 
a  contracting  agency  may  have  failed  to 
carry  out  its  obligations  to  conform  with 
the  entire  treatment  of  solicitation  and 
award  policy  and  procedure. 

Agency  award  decisions  have  been 
overturned  when  GAO  finds  (in  the 
judgment  of  its  legal  staff)  that  the 
agency  failed  to  follow  stated  evaluation 
criteria.  Examples  of  the  bases  for  a 
GAO  decision  to  sustain  a  protest 
include  the  following:  (1)  The  agency 
reviewed  an  appendix  that  contained 
more  pages  than  allowed  by  the 
solicitation  (B-242289,  April  18,  1991); 
(2)  The  agency  record  of  its  decision  did 


5o2 


not  support  its  decision  that  the 
awardee’s  proposal  was  superior  to  the 
protestor’s  (B-242396,  April  29,  1991); 
(3)  The  agency  record  did  not  indicate 
proper  consideration  of  relative  technical 
merit  and  awarded  to  the  low  cost 
offeror,  even  though  technical  factors 
were  stated  to  be  more  important  than 
cost  (B-243357,  July  25,  1991);  (4)  The 
agency’s  record  failed  to  include 
narratives  listing  the  strengths, 
weaknesses,  and  risks  of  protestor’s 
proposal  (B-243051 ,  June  28, 1991);  (5) 
The  agency  evaluation  record  lacked 
substantive  comments  that  supported 
the  reduced  technical  scores  accorded 
protestor’s  proposal  (B-244528.2, 
November  4,  1991). 

These  examples  are  presented  solely 
for  the  purpose  of  illustrating  that  the 
GAO  regularly  substitutes  its  judgment 
for  that  of  the  contracting  officer  or 
agency.  Each  of  the  decisions  cited 
reflects  careful  deliberation  and 
decision,  and  the  rightness  of  the 
individual  decisions  is  not  questioned  in 
this  discussion.  The  question  of  interest 
from  the  perspective  of  this  study  is 
whether  the  current  governmental 
buying  system  actually  operates  to 
achieve  maximum  public  benefit.  It 
seems  important  to  question  issues 
such  as  the  emphasis  on  pre¬ 
determined  criteria  for  evaluation,  the 
demand  for  comprehensive 
documentation,  the  rigid  constraints  on 
communications  between  buyer  and 
seller,  the  existence  of  an  externally 
directed  protest  procedure,  and  the 
extraordinary  detail  embraced  in  law  and 
regulation  so  as  to  govern  the  process 
of  awarding  contracts.  Is  this  process 
justified?  It  is  a  process  that  is  totally 
foreign  to  the  thinking  and  practices  that 
are  proving  to  be  major  elements  in  the 
revitalization  of  American  industry.  The 


research  presented  earlier  in  this  report 
provides  compelling  evidence  that  a 
more  flexible  buying  procedure  brings 
about  greater  levels  of  competitive 
activity  and  greater  public  benefit  than  is 
achievable  with  rigid  objective 
purchasing  procedures  and  policies. 

Summary 

Five  ideas  are  drawn  from  this  study 
as  a  succinct  statement  of  those 
purchasing  practices  that  are  helping  to 
make  top  United  States  companies  more 
competitive  on  a  global  basis. 

1.  Timely  purchasing  decision¬ 
making  coupled  with  optimization 
of  administrative  costs; 

2.  Verification  and  certification  of  a 
potential  supplier’s  quality  and 
reliability  before  allowing  the 
supplier  to  compete  for  an  award; 

3.  Dedication  to  total  cost  analysis 
after  consideration  of  all  costs 
associated  with  the  purchase 
decision  in  place  of  simple  price 
analysis; 

4.  Assessment,  as  part  of  the 
source  selection  process,  of 
supplier  willingness  and  ability 
(flexibility)  to  provide  information 
and  to  coordinate  designs  and 
production  processes  with 
specific  needs  of  the  buyer,  and; 

5.  Measurement  of  performance  with 
recognition  of  superior 
performance  though  continuation 
and  expansion  (by  allocation)  of 
business. 

These  practices  are  becoming 
commonplace  in  industrial  purchasing. 
Although  not  easily  adopted  by 
government  because  of  the  fear  that 
bias  or  other  forms  of  favoritism  might 
reduce  the  fairness  of  the  contract 
award  processes,  these  practices  are 
effective  and  reduce  cost.  Furthermore, 
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they  are  feasible  for  government. 
Protection  of  fairness  and  other 
objectives  can  be  achieved  by 
measures  other  than  restrictive  buying 
procedures. 

Preliminary  Policy  Modification 
Suggestions 

Comprehensive  change  in  the 
volumes  of  regulatory  material  that 
define  government  buying  seems 
unlikely.  However,  some  guidance 
could  be  added  to  existing  policy  that 
would  enhance  the  willingness  of 
administrators  to  exercise  sound 
judgment.  Actions  that  could  be 
adopted  by  the  Congress  follow; 

1.  Adopt  a  broadly  based  policy 
requiring  pre-qualification  of 
potential  suppliers  before  allowing 
submission  of  proposals  for  other 
than  commercial  products  of 
more  than  $100,000.  The  policy 
would  require  agencies,  in 
advance  of  the  release  of 
solicitations,  to  publish  their 
needs  in  the  Commerce  Business 
Daily,  together  with  known 
constraints  or  standards  that 
qualified  suppliers  must  meet. 
Actual  release  of  solicitations  to 
qualified  parties  would  be 
determined  by  the  contracting 
officer  after  considering 
expressions  of  interest.  To  the 
extent  needed,  visitation  at 
suppliers’  facilities  by  a 
government  team  of  experts 
would  precede  release  of 
solicitations. 

2.  Adopt  the  following  standard  for 
entering  into  full  and  open 


competition:  The  intent  of  the 
Congress  in  requiring  full  and 
open  competition  is  to  obtain 
services  and  supplies  for  public 
use  that  constitute  the  best  value 
for  the  public  after  consideration 
of  cost,  price,  and  quaiiiy  factors. 
Agency  contracting  procedures 
shall  be  designed  to  carry  c 't  this 
objective. 

3.  Adopt  the  following  standard  for 
any  d^  -^c^ion  by  an  agency  that 
extraordinary  measures,  such  as 
a  major  investment,  will  be 
initiated  for  the  purpose  of 
creating  full  and  open  competi¬ 
tion:  It  is  the  intent  of  the 

Congress  thrt  agency  efforts  to 
secure  full  and  open  competition 
through  extraordinary  measures 
such  as  public  investment  shall 
be  pursued  only  after  the 
contracting  officer  has  made  a 
finding  and  determination  that  one 
of  the  following  public  objectives 
is  attainable  only  by  adopting  li  ie 
specific  extraordinary  measures: 

a)  after  considering  all  expense 
and  revenue  items  and  finding 
that  a  net  reduction  in  the  overall 
cost  of  the  undertaking  will  be 
achieved  though  adoption  of  the 
extraordinary  measures,  or 

b)  af.er  determining  that 
innovative  or  creative  work  must 
be  acquired,  and  after  reaching  a 
conclusion  that  the  requisite 
innovative  or  creative  effort  can 
be  attained  only  by  adoption  of 
the  proposed  extraordinary 
measures. 
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ABSTRACT 

This  paper  is  part  of  a 
comprehensive  study  involving 
some  three  dozen  major  DOD 
programs  from  all  three 
Services.  The  paper  focuses  on 
schedule  slippage  in  a 
subsample  of  a  dozen  major 
systems.  It  was  found  that  on 
the  average  a  system 
experienced  a  total  of  38 
months  in  schedule  slippage 
before  reaching  Initial 
Operational  Capability,  with 
about  half  of  it  occurring 
during  Engineering 
Manufacturing  Development,  and 
the  rest  of  it  about  evenly 
split  between  Concept 
Demonstration/  Validation  and 
early  Production.  Most 
programs  experience 
significant  additional 
schedule  slippage  beyond  IOC 
because  there  are  usually  more 
systems  in  the  production 
phase  than  we  can  adequately 
fund.  Major  drivers  of 
schedule  slippage  are 
technical  difficulties  and 
fundin2  "vibrations"  -- 


mandated  by  Congress,  OSD 
and/or  the  Services  --  and 
associated  program  changes  and 
restructuring.  Schedule 
slippage  invariably  increases 
program  cost  and  usually 
translates  into  significantly 
shorter  effective  field  life 
of  the  system,  thereby 
providing  a  considerably  lower 
net  return  on  the  tax-payer’s 
dollar. 

INTRODUCTION 
The  time  span  from  the 
point  of  launching  the 
development  of  a  new  system  to 
the  point  where  it  is  put  in 
the  hands  of  an  appropriate 
combat  unit,  thereby  providing 
an  initial  operational 
capability,  is  usually 
referred  to  as  the  length  of 
the  acquisition  cycle.  Growth 
of  the  acquisition  cycle  is 
commonly  called  schedule 
slippage.  Most  major  defense 
programs  experience 
significant  schedule  slippage. 
This  paper  is  based  on  partial 
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results  of  a  comprehensive 
study  of  defense  systems 
acquisition  involving  the 
following  major  programs  from 
all  three  Services:  Patriot, 
Harpoon,  LAMPS,  Bradley,  F-15, 
F-16,  F-IS,  Hellfire,  HARM,  IR 
Maverick,  DSCS  III, 
Peacekeeper,  TOW  2,  Tacit 
Rainbow,  M-1,  Blackhawk, 
Apache,  Chinook,  LHX,  MSE, 
SINGARS,  JTIDS,  ASPJ ,  ADDS, 
T45T,  FDS,  NAVSTAR,  GPS,  DSP, 
DMSP ,  LSD-4  1 ( CV )  ,  CVN-68 ,  CG— 
47,  SSN-66S,  and  DDG-51.  The 
paper  focuses  on  schedule 
slippage,  in  each  phase  of  the 
acquisition  cycle, in  a 
subsample  of  a  dozen  major 
programs . 

WHY  SCHEDULE  SLIPPAGE? 

Why  do  we  often  have 
schedule  slippage?  In  the  most 
general  sense,  because  we  have 
great  difficulty  estimating 
the  cost  and  performance  of 
future  new  developments. 
Obviously,  the  more  complex 
the  new  system  and  the  longer 
the  forecasting  horizon,  the 
less  likely  we  are  to  envision 
all  the  difficulties  w^e're 
going  to  encounter  and 
therefore  the  higher  the 
estimating  error  is  likely  to 
be.  This  is  frequently 
referred  to  as  the  conspiracy 
of  optimism.  But  the  more 
flexibility  the  Program 
Manager  has  in  exercising  his 
best  judgement  regarding 
trade-offs  among  cost, 
schedule  and  p  e  r  f  o  r  m  a  n  c  e  .  the 


smaller  the  ultimate  schedule 
slippage  will  probably  be.  In 
this  connection  it  is  useful 
to  compare  the  defense 
acquisition  environment  w’ith 
that  of  successful  companies 
in  the  commercial  sector.  In 
the  commercial  sector, 
schedule  is  frequently  of 
utmost  importance,  because  the 
magnitude  of  the  profit 
associated  with  the  system  is 
frequently  critically 
dependent  on  the  company’s 
ability  to  market  it  in  a 
particular  time  window. 
Therefore,  here  major  systems 
are  usually  developed  in  a 
highly  streamlined  environment 
where  (1)  the  PM  reports 
directly  to  top  management, 
e.g.,  to  the  head  of  the 
company  or  to  an  executive 
vice  president.  (2)  the  go- 
ahead  to  proceed  to  the  next 
task  is  usually  accompanied  by 
adequate  funding  to  accomplish 
that  task,  and  (3)  the  PM  has 
considerable  flexibility  and 
a  u  t  h  o  r  i  t  >■ . 

useful  example  is  the 
IB.M-360  which  revolutionized 
the  mainframe  computer 
industry.  Here  the  PM  {Bob 
Evans)  reported  directly  to 
the  president  of  the  company 
(Vince  Learson).  traded  off 
performance  for  cost  and 
schedule  by  using  relatively 
c  o  n  s  e  r  V  a  t  1  e  technology  (solid 
logic  technology  or  SLT  vice 
monolithic  integrated 
circuits),  incurred  enormous 
cost  overruTis.  but  "fielded” 
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the  system  in  just  over  four 
years  from  conception. 

By  contrast,  how  long  is 
the  defense  acquisition  cycle 
for  major  weapon  systems? 
According  to  the  Packard 
Commission  Report  it  is 
typically  about  15  years  long. 
And  the  DOD  PM  lives  in  a  much 
more  complex  environment, 
exercising  considerably  less 
authority  and  flexibility. 
Although,  as  a  result  of 
recent  streamlining,  there  are 
now  only  two  management  layers 
between  the  PM  and  the  Defense 
Acquisition  Executive,  there 
are  quite  a  few  players  in  the 
wings,  very  few  whom  can 
provide  the  green  light  or  the 
funding  to  accomplish  a 
particular  task,  but  each  of 
whom  can  inject  a  considerable 
number  of  stumbling  blocks 
into  the  PM ’ s  path. 

Once  the  initial 
estimates  of  the  system's 
performance  parameters  become 
known,  various  defense 
constituencies  --  e.g.,  the 
vulnerability  community,  the 
reliability  community,  the 
logistics  support  community  -- 
become  the  self-appointed 
"guardians"  of  these 
parameters,  making  it 
extremely  difficult  for  the  PM 
to  trade  off  performance  for 
schedule  as  he  inevitably 
encounters  technical 
difficulties.  Trading  off  cost 
growth  is  also  much  more 
difficult  than  in  the 


commercial  sector  because  the 
defense  PM  is  living  in  an 
environment  where  there  are 
already  many  more  programs  in 
the  acquisition  pipeline  than 
we  can  afford  to  fund,  and 
where,  under  the  best 
conditions,  funding  lead  times 
are  usually  18  months  or 
longer.  Quoting  a  paper  by 
F.  Biery,  the  Packard 
Commission  Report  proudly 
cites  the  fact  that  cost 
overruns  in  the  defense 
acquisition  are  typically  a 
small  fraction  of  what  they 
are  in  the  commercial  sector. 
The  other  side  of  the  coin,  of 
course,  is  that  the  defense 
acquisition  cycle  is  more  than 
twice  as  long. 

Why  should  we  be 
concerned  about  the  schedule 
slippage?  Because,  according 
to  the  Packard  Commission 
Report,  "It  leads  to  obsolete 
technology  in  our  fielded 
equipment.  We  forfeit  our 
technological  lead  by  the  time 
it  takes  us  to  get  our 
technology  from  the  laboratory 
into  the  field." 

CONCEPT  EXPLORATION 
The  very  nature  of  the 
Concept  Exploration/ 

Definition  phase  does  not  lend 
itself  to  schedule  slippage. 
The  purpose  of  this  phase  is 
to  translate  the  functional 
system  requirements  into 
hardware  alternatives.  The 
primary  activity  is  the 
para  1  1  e  1 
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management  of  about  half  a 
dozen  small  "study"  contracts 
dealing  with  promising  system- 
design  concepts  for  meeting 
mission  needs.  Each  of  these 
design  concepts  is  explored  by 
its  respective  contractor  in 
great  depth,  with  the  aim  of 
devising  a  concrete  plan  for 
translating  the  concept  into 
hardware  and  for  testing  and 
evaluating  the  critical 

features  of  the  concept  in  the 
next  phase.  There  is  far  too 
little  visibility  at  this 
point  in  the  cycle  into  the 
ultimate  performance  and  cost 
of  the  system  to  entertain 
potential  slippage  in 

schedule.  And,  indeed,  our 
sub-sample  showed  virtually  no 
schedule  slippage  in  this 
phase  . 

DEM/VAL 

Signs  of  potential 
schedule  slippage  begin  to 
appear  early  in  Concept 
Demonstration/Validation  and 
intensify  as  the  system  moves 
down  this  phase.  The  main 
objectives  of  the  Dem/Val 
phase  are  to  verify  that  the 
most  promising  design  concepts 
are  sound,  translate  them  into 
concrete  hardware,  and 

demonstrate  that  the  hardware 
can  be  integrated  into  a  final 
system  that  will  meet  mission 
needs.  Schedule  slippage  here 
is  usually  due  to  such  factors 
as  redefinition  of  the  IOC, 
OSD  or  Service  directed 
restructuring  of  the  program. 


Congress ional ly  mandated 
funding  cuts,  etc.  In  our 
subsample,  the  average 
schedule  slippage  in  this 
phase  was  about  ten  months. 
However,  this  may  be  only  part 
of  the  total  picture. 

Two  forces  are  acting 
in  this  phase  to  partially 
suppress  the  visibility  into 
the  ultimate  schedule 
slippage.  One  of  these  is 
related  to  the  way  this  phase 
is  funded;  the  other,  to  the 
nature  of  competition.  In 
spite  of  the  fact  that  the 
most  common  contract  for 
Dem/Val  is  a  cost- 
reimbursement  type,  most 
contractors  usually  spend 
significant  amounts  of  their 
own  money  (unreimbursed  by  the 
Government)  in  this  phase  -- 
i.e.,  they  frequently  make 
business  decisions  to 
subsidize  this  phase  as  an 
investment  towards  improving 
their  chances  of  winning  the 
EMD  contract  and  thereby 
ultimately  the  production 
contract,  wherein  defense 
contractors  typically  recoup 
their  investment  and  make  some 
profit.  The  immediate  benefit 
of  this  subsidy  to  the 
Government  is  of  course  that 
the  Government  gets  more  in 
research  and  development  work 
than  it  pays  for.  The  downside 
is  that  the  contractor  does 
not  always  feel  obligated  to 
share  with  the  Government  all 
the  information  developed  with 
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the  company’s  own  funds.  This 
is  compounded  by  the  fact  that 
the  two  contractors  are 
competing  on  the  basis  of  two 
different  versions  of  the 
system.  Hence,  surfacing 
information  that  is  indicative 
of  potential  difficulties  in 
EMD  and  Production,  might  be 
construed  as  putting  ones 
company  "unnecessarily"  at  a 
competitive  disadvantage. 

EMD 

With  the  system’s  entry 
into  the  Engineering 
Manufacturing  Development 
phase,  three  key  factors  are 
introduced  into  the 
acquisition  picture:  funding 
shortages,  lack  of 
competition,  and  increased 
visibility  into  the  system’s 
cost  and  performance.  All 
impact  schedule  slippage. 

Prior  to  EMD  the 
expenditures  are  relatively 
small  (typically  a  total  of  5% 
of  the  total  system 
acquisition  cost  is  spent  in 
both  of  the  preceding  phases), 
and  therefore  the  program  is 
relatively  immune  to  funding 
problems.  With  the  systems 
entry  into  EMD,  where  for  the 
first  time  big  money  is 
involved  (25%  of 
the  system’s  acquisition  cost 
is  typically  spent  in  this 
phase),  we  find  that  there  is 


not  enough  money  to  fund  all 
the  programs  in  the 
acquisition  pipeline  and  the 
system  begins  to  experience 
funding  "vibrations"  and 
associated  program 
restructur ing . 

The  lack  of  competition 
stems  primarily  from  the 
relatively  large  expenditures 
associated  with  EMD:  The 
Program  Manager  usually 
narrows  his  alternatives  to  a 
single  system  -  design  concept 
—  and  hence  a  single  prime 
contractor.  A  number  of 
engineering  development  models 
of  the  full  system  are  built 
and  subjected  to  extensive 
developmental  and  operational 
testing  under  conditions  that 
closely  resemble  those  the 
system  would  encounter  in  a 
combat  environment. 

Furthermore  the  acquisition 
strategy  is  updated  and 
considerable  effort  is 
expended  towards  laying  the 
groundwork  for  the  next  phase, 
production  and  deployment. 

The  very  nature  of  the 
activities  in  EMD  provide 
increasing  visibility  into  the 
ultimate  cost,  performance 
and  schedule  of  the  system. 

And  the  absence  of  competition 
removes  most  of  the 
inhibitions  to  surface 
difficulties  in  these  areas  as 
soon  as  they  arise.  Indeed, 
most  of  the  program’s  schedule 
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slippage  occurs  in  EMD.  In  our 
sample  the  systems  experienced 
an  average  schedule  slippage 
in  this  phase  of  about  19 
months  —  or  about  double  that 
experienced  in  Dem/Val.  The 
typical  reasons  were  budget 
cuts,  program  reductions, 
technical  problems,  and  system 
integration  and  testing 
difficulties. 

PRODUCTION 

The  very  nature  of  the 
production  phase  provides  full 
visibility  into  schedule 
slippage.  The  initial  units 
reaching  the  field  are 
subjected  to  extensive 
operational  testing,  the 
results  of  which  are  fed  back 
into  the  production  line  and 
used  to  refine  the  methods  of 
production,  improve  the 
subsequent  lots  of  the  system 
and  optimize  the  operational 
concepts.  Virtually  all 
systems  in  our  sample 
experienced  considerable 
difficulty  in  transitioning 
from  EMD  to  Production.  This 
difficulty  translated  into  an 
average  schedule  slippage  of 
nine  months  in  this  phase. 
Furthermore,  most  systems 
experienced  significant 
additional  schedule  slippage 
--beyond  IOC  and  after 
product  ion  had  been 
"stabilized"--  because  of  a 
lack  of  funding  to  accommodate 
all  programs  in  this  phase. 


SUMMARY 

The  DOD  Program  Manager 
functions  in  a  much  more 
complex  environment  than  his 
counterpart  in  the  commercial 
sector.  Thus  he  finds  it  much 
more  difficult  to  trade  off 
performance  and  cost  for 
schedule.  Since  initial 
estimates  of  system 
performance  parameters  are 
inevitably  optimistic,  this 
leads  to  more  schedule 
slippage  and  a  much  longer 
acquisition  cycle.  In  our 
sample,  about  half  the  total 
schedule  slippage  occurred  in 
Engineering  Manufacturing 
Development  phase,  with  the 
rest  approximately  equally 
divided  between  Concept 
Demonstration/Validation  and 
early  Production.  Beyond  IOC, 
systems  usually  incur 
considerable  additional 
schedule  slippage  because 
there  are  more  programs  in  the 
defense  acquisition  pipeline 
than  we  can  adequately  fund. 
Schedule  slippage  is  one  of 
our  worst  enemies  --  because 
it  robs  our  systems  of  much  of 
their  technological  lead. 
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BRIDGING  THE  GAP  -  FROM  REQUIREMENT  DEFINITION  TO  CONTRACT  AWARD 


Bonnie  D.  Taylor,  Procurement  Analyst 
U.S.A.F  (Wright  Laboratory) 


Abstract  -  Wright  Laboratory,  which 
is  the  largest  of  the  Air  Force 
Materiel  Command's  four  "superlabs", 
has  been  at  the  forefront  of  major 
cl'nges  in  technology  acquisition 
over  the  past  three  years.  Their 
success  story  has  been  briefed  to 
Army  and  Navy  activities  as  well  as 
industry  associations  and  other  Air 
Force  activities  with  overwhelmingly 
positive  feedback.  The  initiatives 
which  resulted  in  dramatic  improve¬ 
ments  to  acquisition  leadtime  were 
accomplished  during  a  period  of 
decreasing  manpower  with  no  compara¬ 
ble  reduction  in  workload;  and  they 
were  accomplished  with  existing 
resources  and  no  additional  cost  to 
the  taxpayer. 

The  improvements  affected  teamwork, 
the  acquisition  process  itself, 
management  and  software  tools  which 
were  developed  in-house  for 
consistent  and  efficient  document- 
tion.  The  changes  weren't  magic  and 
the  results  were  not  as  sudden  as  a 
lightning  strike,  but  the  increase 
in  morale  and  professional  pride  was 
just  as  dramatic.  The  beginning  of 
the  WL  story  of  continuous  improve¬ 
ment  and  the  details  of  the  initia¬ 
tives  that  have  made  it  happen  are 
summarized  here. 

Introduction  -  Technology  is  not  a 
natural  evolution.  Our  ancestors' 
discovery  of  fire  and  development  of 
simple  tools  for  survival  was 
significantly  different  than  the 
advancements  in  the  state  of  the  art 
needed  today.  In  order  to  establish 
a  secure,  national  defense,  to  be 
competitive  on  an  international 
level  or  to  promote  the  general 
welfare,  time  is  critical.  The 
longer  it  takes  to  begin  the 
development  effort,  the  smaller  the 


window  for  effective  transition  of 
the  resulting  technology.  The 
administrative  leadtime  over  all 
four  stages  of  development  from 
basic  research  through  full  scale 
development  can  add  years  to 
fielding  or  improving  weapon  sys¬ 
tems.  As  a  result,  leadtime 
measurement  has  become  increasingly 
important; and  laboratories  are 
competing  for  diminishing  research 
dollars  by  stripping  the  frills  from 
the  acquisition  process.  Within 
DOD,  technical  excellence  and  a 
quick  turnaround  for  an  inter-agency 
purchase  request  means  increased 
business  for  a  laboratory. 

Over  the  past  three  years,  Wright 
Laboratory  (WL)  has  pursued  the  "no 
frills"  approach  and  actively 
managed  the  process  to  eliminate 
scrap  and  rework  with  remarkable 
success.  Some  of  the  initiatives 
were  driven  by  leadtime  statistics 
which  showed  a  dramatic  need  to 
improve;  others  were  the  result  of 
participation  in  the  DOD  Laboratory 
Demonstration  Program  and  joint 
conferences.  Many  were  developed 
in-house  through  dedicated  Total 
Quality  Management  teams  and 
independent  brainstorming  sessions, 
but  borrowing  and  sharing  with  the 
other  laboratories  also  played  a 
critical  role.  The  overall  goal  for 
all  Wright  Laboratory  personnel 
was  to  bring  the  business  reputation 
up  to  the  same  high  standards  as  the 
technical  reputation  and  to  develop 
technology  in  a  more  timely  manner. 

The  resulting  initiatives  included 
early  development  of  acquisition 
teams  and  user  involvement; 
management  commitment  and  focus; 
industry  participation  in  process 
improvement;  measurement  and 
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accountability;  maximizing  competi¬ 
tion  through  increased  use  of 
Program  Research  and  Development 
Announcements  (PRDA)  and  Broad 
Agency  Announcements  (BAA);  improved 
communications  with  industry  and 
within  the  laboratory;  an  effective 
small  business  program  and  stream¬ 
lined  Small  Business  Innovation 
Research  awards;  as  well  as  the  use 
of  dutomated  tools  and  electronic 
media  to  improve  the  process.  The 
combined  effect  was  a  leadtime 
reduction  of  47%  for  competitive 
source  selections;  45%  for  PRDA/BAA 
awards;  and  81%  for  Phase  I  awards 
under  the  Small  Business  Innovation 
Research  Program.  The  time  saved  in 
acquisition  has  been  directly 
proportional  to  customer  satisfac¬ 
tion  and  mission  support.  The  "hows" 
and  "whys"  of  each  of  the  major 
initiatives  are  summarized  on  the 
following  pages. 

Teamwork  -  One  of  the  biggest 
hurdles  in  the  early  stages  was  the 
concept  of  "us"  vs  "them".  Each 
of  the  players  in  the  acquisition 
process  was  committed  to  excellence 
while  the  baton  was  in  his/her  hand; 
but,  once  passed,  the  acquisition 
was  someone  else's  responsibility. 
This  assembly  line  approach  failed 
to  recognize  the  contribution  of 
each  member  of  the  team  throughout 
the  process.  We  turned  this 
thinking  180  degrees  around  by 
involving  team  members  with  early 
kick-off  meetings  and  improved 
communication  between  the  contract 
negotiator/contracting  officer,  the 
financial  manager,  and  the  project 
engineer . 

The  change  in  cultural  philosophy 
didn't  happen  overnight.  In  1989  a 
team  of  WL  senior  managers  visited 
Phillips  Laboratory  to  identify 
procedures  and  practices  that  could 
be  implemented  for  improvements  at 
home.  The  most  significant  lesson 
was  teamwork  and  commitment  to 


schedule . 


Based  upon  the  lessons  learned,  new 
milestone  schedules  were  negotiated 
under  the  WL  Commander's  direction 
to  include  pre-  and  post-  purchase 
activities.  The  new  schedules  and 
the  new  team  strategy  were  briefed 
to  all  laboratory  personnel  through 
a  series  of  acquisition  seminars  in 
the  fall  of  1989.  The  all-day 
seminars  were  designed  to  present  a 
total  picture  of  the  acquisition 
process  so  team  members  could  better 
understand  and  relate  to  each 
other's  respective  roles  and 
requirements.  Topics  included 
statement  of  work  preparation, 
evaluation  criteria,  the  source 
selection  process  and  other 


streamlining 
within  WL. 


initiatives  underway 


Top  level  management  attention  was 
focused  on  milestone  reviews  with 
mixed  results.  Leadtimes  improved 
significantly,  but  the  emphasis  on 
time  was  occasionally  misunderstood 
at  the  cost  of  quality  in 
documenting  the  acquisition  process. 
The  Air  Force's  implementation  of 
the  clearance  process  the  following 
year  added  to  the  buyer's 
frustration  by  accomplishing  reviews 
at  earlier  milestones  during  the 
acquisition.  The  adage  that 
"Quality  is  quicker"  was  a  major 
cultural  change  for  many,  but  the 
success  stories  of  proactive  teams 
provided  the  proof  of  concept. 

Without  exception,  the  early 
involvement  of  all  team  members, 
review  of  draft  documents,  continu¬ 
ous  communication  during  the  process 
and  the  commitment  of  everyone  to 
the  negotiated  milestones  resulted 
in  high  quality  documentation  and 
record  setting  times.  The  contrast 
was  dramatic  between  programs 
which  built  in  quality  by  utilizing 
ad/isors  throughout  the  process  and 
program  dependent  upon  inspection 
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and  review  at  selected  milestones. 
The  leadtime  from  submission  of  the 
request  for  business  clearance 
through  resolution  of  issues  and  the 
recommendation  for  approval  varied 
from  2  to  94  days  for  the  extreme 
scenarios.  The  milestone  reviews 
which  focused  on  these  variances 
provided  feedback  to  management  to 
initiate  training  and  foster 
accountability.  The  success  came 
full  circle  in  1992  when  senior 
managers  from  other  DOD  and  industry 
organizations  visited  WL  to  identify 
practices  that  would  help  them 
achieve  the  same  results. 

Process  Improvement  -  Developing  an 
awareness  and  understanding  of  the 
process  to  be  improved  is  one  of  the 
most  challenging  stages  of  total 
quality  management.  A  critical 
process  team  *  as  formed  by 
contracting,  financial  and  technical 
managers  in  January  1989.  The  team 
analyzed  each  of  the  tasks  required 
for  acquisition  of  a  program  using  a 
competitive  RFP.  The  task  breakdown 
structure  was  diagrammed  on  a 
critical  path  chart  which  covered 
26  feet  of  acquisition  activity. 
The  chart  was  tangible  evidence  of 
the  complexities  of  the  process,  but 
it  also  provided  the  visibility  for 
targeting  high  payoff  areas.  Thirty 
individual  corrective  action  teams 
(CATs)  involving  over  80  people, 
were  ultimately  formed  to  follow-up 
with  initiatives  in  each  of  these 
areas . 

Recognition  of  customers  was 
important  in  forming  each  CAT  and 
the  inclusion  of  both  engineers  and 
contract  specialists  on  each  team 
was  the  obvious  first  step.  As  the 
functional  members  expanded  their 
outlook  on  the  acquisition  picture 
to  include  t)  j  roles  played  by  other 
team  members,  it  became  equally 
clear  that  the  26  feet  emphasized 
Government  responsibilities  without 
providing  the  same  level  of  detail 


for  the  other  side  of  the  process. 
In  1991  WL's  annual  R&D  Contracting 
Day  with  Industry  included  represen¬ 
tatives  from  the  National 

Security  Industrial  Association 
(NSIA)  in  the  development  of 
material  to  be  presented  and  in  the 
seminar  itself.  The  overwhelmingly 
favorable  feedback  led  to  industry 
representatives'  joining  the  WL 
critical  process  team  (CPT) . 

These  external  customers,  volunteers 
from  the  NSIA  with  a  relatively 
broad  base  of  acquisition  experience 
and  a  high  level  of  interest  and 
enthusiasm,  were  able  to  make 
important  contributions  to  the 
streamlining  process.  Within  the 
first  two  years  the  success  stories 
of  the  critical  process  team 
resulted  in  over  thirty  policy  or 
procedure  changes  for  WL.  One  of 
he  most  visible  was  the  reduction 
of  internal  operating  instructions 
from  53  separate  documents  to  17. 
By  taking  a  critical  look  at 
practices  developed  in-house  over 
long  periods  of  time,  we  were  able 
to  eliminate  many  of  the 
adminstrative  burdens  we  had  imposed 
upon  ourselves  for  problems 

which  no  longer  existed. 

The  teams  also  generated  process 
improvements  affecting  regulation 
implementation  (eliminating  the 
scrap  and  rework  caused  by 
regulatory  changes  after  RFP 
release);  Contract  Data  Requirements 
List  preparation  (automating  the  DD 
Form  1423  and  minimizing 

coordinations);  RFP  streamlining 
(allowing  concurrent  release  and 
review);  and  implementing  a  more 
timely  literature  search.  Actions 
within  the  laboratory's  control 
resulted  in  rapid  implementation  of 
change.  Many  initiatives  requiring 
higher  levels  of  approval  have  also 
been  implemented  and  others  are 
being  actively  pursued. 
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The  selection  of  WL  as  one  of  the 
DOD  Demonstration  Laboratories  in 
March  1991  provided  the  platform  to 
pursue  more  wide-spread  activity. 
The  goal  of  the  DOD  Lab  Demo  program 
is  to  increase  the  productivity  and 
effectiveness  of  the  DOD  laborato¬ 
ries  through  improvements  in  person¬ 
nel  management,  research  related 
contracting,  facilities  refurbish¬ 
ment  and  management  authorities. 

Over  60  streamlining  initiatives 
were  generated  within  WL  in  four 
separate  areas:  In-house,  Person¬ 
nel,  Procurement  and  Management. 
The  In-house  initiatives  dealt 
primarily  with  the  acquisition  of 
materials  and  equipment  necessary  to 
support  the  on-going  activities  of 
the  laboratory;  personnel  issues 
were  related  to  hiring  authority  and 
classif ication;and  management  ini¬ 
tiatives  involved  realignment  of 
some  functional  support  organiza¬ 
tions,  including  management  report¬ 
ing,  and  improvements  in  processing 
travel  requests.  Procurement  ini¬ 
tiatives  affected  contracting, 
financial  and  program  management 
personnel,  eliminating  and  stream¬ 
lining  documentation.  Autonomy  was  a 
key  thrust  in  many  of  these 
initiatives,  providing  increased 
delegations  of  authority  for 
acquisition  strategy  planning, 
source  selection  decisions,  and  for 
financial  management  to  the 
Laboratory  Commander.  By  assigning 
program  accountability  and  responsi¬ 
bility  to  the  laboratory  managers, 
R&D  decisions  are  made  by  those  with 
the  most  knowledge  and  expertise  in 
Science  and  Technology. 


One  of  the  most  dynamic  changes 
proposed  for  the  Lab  Demo  Program  is 
undergoing  review  and  revision.  The 
initial  draft  Included  streamlined 
solicitation  procedures  and  a 
simplified  R6D  contract  format.  The 
proposal  was  based  in  part  on 
the  current  process  for 
awards  in  response  to  Program 


Research  and  Development  Announce¬ 
ments  (PRDAs)  and  Broad  Agency 
Announcements  (BAAs).  The  approach 
would  allow  publication  of  the 
solicitation  in  the  Commerce 
Business  Daily,  source  selection 
based  upon  standard  or  streamlined 
procedures  and  award  using  an 
abbreviated  contract  document.  This 
Tri-Service  initiative  demonstrates 
the  importance  of  improved  R&D 
contracting  throughout  DOD. 

A  change  in  philosophy  for 
competitive  range  determinations  was 
a  significant  outgrowth  of  the  Lab 
Demo  Program.  Regulatory  language, 
implemented  over  time,  had 
influenced  most  source  selection 
teams  to  retain  all  offerors  with  a 
technically  acceptable  rating  of 
marginal  or  above  in  the  competitive 
range.  An  analysis  of  source 
selection  records  within  WL  showed 
that  over  90%  of  the  awards 
resulting  from  competitive  Requests 
for  Proposals  were  made  to  the 
offeror  with  the  highest,  initial 
technical  rating.  In  some  cases  the 
highest  technical  offeror  may  have 
also  been  the  lowest  proposed  price, 
but  the  importance  of  technical 
merit  in  research  proposals  became 
very  clear.  Numerous  phone  calls 
and  several  trips  to  Washington  with 
excellent  support  from  headquar¬ 
ters  personnel  produced  the 
necessary  change  in  Air  Force 
regulations.  Competitive  range 
determinations  still  give  every 
offeror  the  benefit  of  the  doubt, 
but  only  those  with  a  reasonable 
chance  for  award  are  retained 
through  discussions  and  the  request 
for  Best  and  Final  Offer  (BAFO). 

At  about  the  same  time,  legislative 
action  resulted  in  increased 
opportunities  for  award  without 
discussion.  Statutory  language  was 
changed  to  recognize  both  price  and 
other  factors  in  the  decision  to 
award  without  discussions.  A  source 
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selection  decision  to  other 
than  the  low-priced  offeror  no 
longer  forces  the  acquisition  team 
into  discussions  and  BAFO.  The 
importance  of  an  offeror's 
submitting  the  best  terms  from  a 
price  and  technical  standpoint  with 
the  initial  offer  has  been 
emphasized  in  the  solicitations  and 
in  briefings  to  industry.  The  use 
of  award  without  discussions  has 
increased  from  less  than  1%  to  more 
than  20%  over  the  past  two  years, 
reducing  administrative  costs  and 
time  for  both  the  Government  and 
industry . 

Increased  use  of  PRDAs  and  BAAs  has 
also  been  effective  in  reducing 
acquisition  leadtime.  The  PRDA  was 
implemented  in  1982  on  a  test  basis 
to  encourage  broader  participation 
by  science  and  technology  based 
firms  in  meeting  research  and 
development  goals  as  well  as  to 
decrease  contracting  lead  times. 
Although  the  PRDA  was  initially 
developed  to  identify,  publicize  and 
encourage  proposal  submissions  for 
basic  research  and  exploratory 
development,  its  use  as  a 
streamlined  competitive  procedure 
has  varied  based  upon  different 
product  center  and  laboratory 
interpretations.  An  Air  Force 
Systems  Command  Joint  Directors 
Conference  in  1989  identified 
increased  use  of  the  BAA/PRDA 
process  as  a  Science  and  Technology 
initiative.  As  a  result,  WL's  use 
of  the  PRDA  changed  significantly. 
The  percentage  of  competitive 
actions,  excluding  Small  Business 
programs,  awarded  using  PRDA  or  BAA 
procedures  increased  from  49%  in 
1990  to  70%  in  1992  and  the  leadtime 
for  PRDA/BAA  awards  was  reduced  from 
134  days  in  Fy89  to  74  days  in  FY92. 

The  effectiveness  of  WL's  small 
business  program  has  also  had  a 
positive  impact  on  technology 
implementation.  The  Small  Business 


Innovation  Research  (SBIR)  program 
was  mandated  by  Public  Laws  97-219 
and  99-443  to  stimulate  techno¬ 
logical  innovation,  strengthening 
the  role  of  small  business  in 
meeting  DOD  research  and  development 
needs,  fostering  and  encouraging 
participation  by  minority  and 
disadvantaged  persons  in  technologi¬ 
cal  innovation,  and  increasing  the 
commercial  application  of  DOD- 
supported  research  or^  research  and 
development  results.  Accomplish¬ 
ment  of  these  goals  is  partially 
measured  by  how  effectively  the 
program  is  managed  and  the  timeli¬ 
ness  of  processing  awards.  By 
establishing  dedicated  teams  for 
SBIR  negotiations,  improving  commu¬ 
nications  and  milestone  commitments 
from  the  entire  team,  and 
streamlining  cost  proposal  analysis, 
the  leadtime  for  SBIR  awards  was 
reduced  from  88  days  in  1989  to  17 
days  in  1992.  This  reduction 
reflects  a  highly  proactive  and 
concurrent  approach  between 
contracting  and  technical  personnel. 

In  addition  to  improvements  in  the 
SBIR  process,  WL  has  recognized  the 
importance  of  the  small  business 
contribution  to  technology  develop¬ 
ment  and  transition  through  a 
well-defined  and  aggressive  small 
business  program.  In  1992  WL  was 
the  first  Federal  Component  of  any 
agency  to  receive  a  Small  Business 
Program  Effectiveness  Rating  of 
Exceptional  by  the  Small  Business 
Administration.  The  surveillance 
report  noted  "efforts  beyond  the 
regulatory  requirements  to  achieve 
the  purposes  of  the  small  business 
program"  and  the  resulting  accolades 
attributed  the  success  to  the 
environment  of  trust,  empowerment 
and  teamwork  created  by  WL,  the 
Small  Business  Program  office  and 
the  Command.  Training,  management 
emphasis  and  improved  communication 
have  all  contributed  to  an  open 
atmosphere,  eliminating  much  of  the 
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mystique  and  time  consuming  red-tape 
of  Government  acquisition. 

Management's  Role  -  The  pressure  to 
work  "smarter  and  faster"  followed 
the  normal  pattern  of  gravity, 
imposing  the  heaviest  burden  on  the 
lowest  tier  in  the  organizational 
structure.  The  increased  responsi¬ 
bility  for  contract  negotiators  had 
the  potential  for  becoming  a 
backlash  of  frustration  and 
unproductive  grumbling,  but  manage¬ 
ment  was  already  adapting  to  the  new 
needs  of  the  organization.  Good 
managers  are  made,  not  born; 
however,  most  supervisors  have  a 
limited  amount  of  training  in 
managing  people.  The  need  to 
balance  management  styles  and  to 
view  employees  as  a  customer  is 
foreign  to  many  supervisors  whose 
performance  is  judged  by  the 
productivity  of  their  unit.  To 
counter  pre-conceived  notions  and  to 
establish  a  universal  culture  and 
philosophy,  senior  managers  in  the 
contracting  organization  initiated  a 
series  of  offsite  working  sessions 
to  identify  clear  goals  and  expected 
management  behaviors. 


Focusing  on  positive  and  productive 
actions,  managers  have  made  a 
concerted  effort  to  promote 
sincerity  and  honesty  in  inter¬ 
actions;  to  be  enthusiastic  about 
the  work  and  its  importance;  to 
treat  people  with  respect  and  insure 
fair  and  equal  treatment;  to  provide 
the  freedom  for  employees  to  take 
risks  and  make  mistakes  in  the 
interest  of  continuing  improvement; 
to  be  interested  and  supportive;  and 
to  give  clear  and  meaningful 
feedback.  As  a  result,  these 
behaviors  have  become  the  organi¬ 
zational  way  of  life  rather  than 
simply  "motherhood  and  apple  pie" 
public  relations  statements. 


Tools  -  Using  resources  most 
effectively  is  critical  to  any 


streamlining  effort  where  "doing 
more  with  less"  is  increasingly 
challenging.  Automation  has  been 
the  solution  for  many  of  these 
seemingly  impossible  demands.  This 
was  due,  in  part,  to  internal 
classes  on  commercial  spreadsheet 
and  word-processing  programs,  but 
the  most  significnat  contribution 
has  been  the  development  of  unique 
programs  in-house  to  automate  the 
preparation  of  acquisition  plans, 
purchase  request  packages,  price 
negotiation  memoranda,  and  contract 
writing  tools.  Tailored  for  R&D 
acquisitions,  these  programs  have 
enhanced  efficiency  by  as  much  as 
30%  to  50%  and  have  improved  overall 
quality. 

Summary  -  The  Council  on  Competi¬ 
tiveness  issued  a  report  in  1992 
which  said  that  most  of  the  federal 
laboratories  are  technology-driven, 
concentrating  on  developing  enabling 
technologies  which  ar^  long-term 
responses  to  problems.  This  was 
in  conflict  with  private  industry 
which  is  market-driven  to  look  for 
short-term  responses.  Although  the 
federal  laboratories  must  continue 
to  focus  on  the  priorities  of  their 
customers,  Wright  Laboratory's 
success  in  reducing  acquisition  lead 
time  will  reduce  the  cultural  gap 
between  the  federal  and  private 
sector.  By  eliminating  administra¬ 
tive  delays,  the  Government  will  put 
the  "rubber  on  the  ramp"  that  much 
sooner,  making  the  technology 
available  for  both  military  and 
commercial  application. 

The  first  step  toward  improvement 
was  recognition  of  the  problem;  the 
second  was  creating  an  environment 
receptive  to  change;  but  the  third 
step  was  the  most  critical.  Without 
management  commitment  and  oversight, 
continuous  improvement  becomes  an 
avalanche  of  good  ideas  without 
successful  mplementation .  Leadership 
is  the  successful  bridge  for  ideas 
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that  change  the  future. 
NOTES ; 


1.  Memorandum  for  Secretaries 
of  the  Military  Departments  Under 
Secretary  of  Defense  for  Acquisition 
(20  Nov  1989) 

2.  DOD  Small  Business 
Innovation  Research  Program 
Solicitation  93.1  {Washington  D.C.: 
United  States  Department  of  Defense, 
Small  Business  Innovation  Reseach 
Program  Office) 

3.  Ronald  E.  Yates;  Refitting 
Cold  War  Science  (Chicago  Tribune; 
26  Oct  1992) 
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Image  Management  and  Acquisition  for  the  Future 

by 

Thomas  J.  Thiel,  Information  and  Image  Management  Technologies 


ABSTRACT 

Image  management  technologies  are  matur¬ 
ing  at  an  ever-accelerating  pace.  Enabled  by 
newly  emerging  optical  storage,  imaging  is 
expected  to  become  to  a  multi-billion  dollar 
industry  by  1995.  Does  imaging  relate  to  the 
reality  of  the  acquisition  office  of  the  future? 

This  presentation  provides  an  overview  of 
electronic  document  image  management  sys¬ 
tems  including  scanning  from  analog  paper 
to  electronic  form,  indexing,  storage,  retriev¬ 
al,  and  the  recreation  of  an  analog  image 
from  the  electronic  format.  The  article  in¬ 
cludes  a  discussion  of  today's  imaging  sys¬ 
tems,  from  low-end  imaging  systems  to 
enterprise-wide  integrated  systems. 

The  second  half  of  the  article  describes  the 
application  of  electronic  imaging  to  a  con¬ 
tracting  example  from  an  unnamed  real- 
world  contracting  office  in  the  federal  gov¬ 
ernment.  It  discusses  the  existing  situation 
in  the  contracting  office  and  the  problems  to 
be  addressed  by  an  imaging  implementation. 
Two  imaging  alternatives,  including  stand 
alone  and  networked  organzation-wide  sys¬ 
tems,  are  presented.  The  impacts  each  of 
these  alternatives  would  have  on  each  of  the 
problem  areas  is  also  presented.  Workflow, 
the  ability  to  route  and  to  add  value  to  a  con¬ 
tract  file's  documents  electronically  is  also 
described. 

Finally,  a  strategy  for  implementing  imaging 
in  the  contracting  environment  is  presented. 
To  most  effectively  use  imaging  technology, 
imaging  systems  should  be  integrated  into  an 
organization's  overall  ADP,  office 


automation,  and  information  management 
systems.  Other  considerations  in  implement¬ 
ing  imaging  includes  the  following:  become 
imaging  smart,  let  business  and  work  consid¬ 
erations  drive  the  imaging  decision,  involve 
the  staff  in  the  process,  create  a  flexible  sys¬ 
tem  capable  of  adapting  to  change  and  new 
applications,  and  implement  work  process 
changes  to  take  advantage  of  imaging. 

IMAGING  OVERVIEW 

Imaging  is  one  of  the  most  important 
technologies  of  the  1990s.  With  imaging  we 
can  deliver  superior  service  to  our  customers, 
improve  information  and  knowledge  pro¬ 
cessing  and  management,  and  achieve  higher 
quality  output  with  fewer  resources.  Imag¬ 
ing  helps  us  to  gain  a  strategic  advantage 
over  our  competitors,  to  ensure  our  organiza¬ 
tional  survival  in  an  increasingly  competitive 
marketplace,  and  to  focus  on  business  pro¬ 
cess  transformations.  This  author  has  pre¬ 
viously  discussed  the  use  of  optical  disk 
systems  for  acquisition  (1);  in  this  article,  a 
brief  overview  of  imaging  is  presented.  The 
basic  components  of  imaging  are  related  to 
the  contracting  activity  in  a  following 
section. 

Imaging  is  defined  as  the  electronic  manage¬ 
ment  of  information  using  digital  representa¬ 
tions  of  documents;  these  digital 
representations  are  called  images.  This  defi¬ 
nition  generally  applies  to  document  image 
management  systems  rather  than,  for  exam¬ 
ple,  machine  vision  and  medical  picture  ar¬ 
chiving  systems,  which  are  referred  to  as 
High  End  Image  Processing  Systems. 
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An  optical  disk  electronic  document  imaging 
system  consists  of  several  or  all  of  the  fol¬ 
lowing  integrated  technologies:  data 
management,  electronic  document  image 
management,  multimedia,  text  retrieval, 
and/or  workflow.  The  goal  of  imaging  is  to 
enhance  and  preserve  the  value  of  an  orga¬ 
nization’s  information  resources,  and  to  opti¬ 
mize  and  streamline  the  organization’s 
business  and  work  functions. 

In  an  imaging  system,  documents  are  stored, 
transmitted,  and  accessed  on-line  via  a  com¬ 
puter  workstation.  Most  electronic  docu¬ 
ment  imaging  systems  use  optical  media 
such  as  write-once  read-many  (WORM)  and 
rewritable  optical  disks.  Compact  Disc  Read 
Only  Memory  (CD-ROM),  and  videodiscs 
for  the  storage  of  electronic  documer  be¬ 
cause  they  have  extremely  high  storage 
capacities. 

A  typical  imaging  system  includes:  an  opti¬ 
cal  disk  drive,  a  computer;  a  high  resolution 
monitor,  a  raster  or  bit-mapped  scanner, 
image  compression  and  decompression 
boards,  a  laser  printer,  and  user  interface  and 
document  indexing  and  retrieval  software  (a 
PC-based  imaging  system  is  shown  in 
Figure  1).  A  system  like  that  shown  in  Fig¬ 
ure  1  might  cost  about  $30,000  from  the 
GSA  Schedule. 

\ 


PC  WORKSTATION 


LASER  PRINTER 

Figure  I.  PCI-Basied  flptical  Disk  F.IrcIronIc  Documcnl 
Imaftr  Managrtncnl  .Syslrm. 


Document  imaging  is  an  automated  electron¬ 
ic  means  of  storing,  retrieving,  transmitting, 
processing,  and  managing  documents,  i.e., 
those  pieces  of  paper  that  fill  file  folders  and 
cabinets.  Electronic  documents  can  be  dis¬ 
played  on  a  computer  monitor  or  printed  on 
paper  in  seconds,  where  and  when  they  are 
needed.  One  of  the  great  advantages  of  an 
imaging  system  is  its  ability  to  capture,  re¬ 
trieve  and  transmit  all  types  of  information, 
including,  handwritten,  machine  created,  dia¬ 
grams,  forms,  photographs,  and  more  recent¬ 
ly,  even  audio,  and  still  and  motion  video 
information. 


Scanning;  Bit-mapped  versus  Analog  and 
Character  Information 

Imaging  preserves  the  visual  and  spatial 
characteristics  and  appearance  of  the  original 
paper  document.  The  information  universe 
primarily  consists  of  analog  information  — 
human-readable  information  on  paper.  It  is 
relatively  easy  to  visualize  this  form  of  in¬ 
formation;  after  all  we  have  been  dealing 
with  paper  for  centuries.  Analog  informa¬ 
tion  is  any  information  that  is  marked  or 
scribed  on  a  medium,  such  as  paper,  film, 
billboards,  or  even  the  walls  of  a  cave  (this 
page  is  an  example  of  the  analog  form). 

On  the  other  hand,  the  computer  stores  and 
manages  information  as  characters  or  as  a 
raster  bit-map  of  analog  information.  All 
analog  information  must  be  converted  to  one 
of  these  two  forms  for  it  to  be  entered  into  a 
computer.  Visualizing  computer  information 
is  more  difficult  than  visualizing  analog  in¬ 
formation.  Additionally,  the  differences  be¬ 
tween  character  and  bit-mapped  forms  of 
computer  information  is  very  significant  and 
one  of  the  most  misunderstood  aspects  of 
document  imaging  systems. 
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With  character  information  (word  processing 
and  databases  for  example),  each  letter  or 
number  is  represented  by  a  single  combina¬ 
tion  of  8  bits  (zeros  or  ones).  ASCII  code  is 
one  form  of  character-based  computer  in¬ 
formation.  For  each  character  on  the  page, 
there  is  a  corresponding  8  bit  binary  repre¬ 
sentation.  That  is  not  the  case  for  bit¬ 
mapped  information. 

In  order  to  bit-map  a  document,  the  docu¬ 
ment  must  be  "scanned,"  or  passed  under  a 
device  called,  logically  enough,  a  scanner. 
The  scanner  "looks"  at  a  page  and  divides  it 
into  a  very  large  number  of  extremely  small 
elements,  called  picture  elements  (also  pixels 
or  dots).  For  each  pixel,  the  scanner  deter¬ 
mines  whether  the  difference  between  that 
pixel's  appearance  and  the  page's  background 
color  is  beyond  a  certain  darkness  level 
(threshold).  If  it  is  beyond  this  level,  the 
scanner  declares  the  pixel  is  part  of  the  ana¬ 
log  information  on  the  paper  and  records  the 
pixel  in  as  a  digital  one.  Those  pixels  that 
are  indeed  background  or  are  not  sufficiently 
dark  to  reach  the  threshold  level  are  recorded 
as  a  digital  zero.  A  representation  of  bit¬ 
mapped  information  is  shown  in  Figure  2. 

^  \ 


F4:ure  2.  Scanned  or  Bit-Mapped  Information 


In  order  to  preserve  the  visual  and  spatial 
characteristics  and  appearance  of  the  original 
paper  document,  imaging  systems  store  the 
document  in  bit-mapped  form.  Any 


information  on  a  document  page,  including 
text,  graphics,  hand  written  signatures  and 
notes,  photographs,  etc.,  can  be  scanned  and 
recorded  in  bit-mapped  form. 

Storage  of  Bit-mapped  Document 
Information 

An  imaging  system  typically  uses  optical 
disk  storage;  in  fact  the  optical  disk  is  one  of 
the  enabling  technologies.  This  is  because  a 
bit-mapped  page  requires  a  very  large  num¬ 
ber  of  ones  and  zeros  to  describe  it.  The 
computer  storage  needed  to  describe  a  bit¬ 
map  of  a  page  is  a  function  of  the  pixel  (or 
scanning)  density  and  to  some  extent  the  in¬ 
formation  on  the  page.  Scanning  densities  of 
200  or  300  dots  per  inch  (dpi)  in  both  the 
horizontal  and  vertical  dimension  are  typical. 
For  example,  describing  an  8.5  by  11 -inch 
office  document  at  a  resolution  of  300  dpi 
requires  8,415,000  ones  and  zeros 
(300  X  8.5  X  300  x  1 1).  Fortunately,  there 
are  computer  algorithms  to  reduce 
(compress)  this  to  a  manageable  level  and 
the  typical  bit-map  storage  requirement  for 
this  page  in  compressed  form  is  approxi¬ 
mately  50,000  bytes. 

Optical  disk  storage  is  generally  used  with 
imaging  because  it  provides  massive  storage 
capabilities,  is  safe  (only  a  laser  light  touches 
the  spinning  optical  disk),  and  is  removable. 
WORM  and  rewritable  optical  disks  are  the 
two  primary  types  used  (Compact  Disc  Read 
Only  Memory  or  CD-ROM  is  also  is  begin¬ 
ning  to  be  used  for  some  applications). 

WORM  disks  come  from  the  supplier  as 
blank  formatted  optical  disks.  A  user  can 
write  any  binary  information  on  any  specific 
position  on  the  disk  one  time  and  only  one 
time.  WORM  disks  cannot  be  erased  and 
rewritten.  Rewritable  optical  disks  are  also 
blank  optical  disks  that  a  user  can  write  any 
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binary  information  to  but  these  optical  disks 
can  be  erased  nd  rewritten  much  like  mag¬ 
netic  disks.  Both  of  these  are  written  opti¬ 
cally,  that  is  with  a  laser  beam.  Since  the 
laser  beam  is  microscopic,  the  recording  den¬ 
sity  is  very  high  thereby  providing  a  very 
large  storage  capability  (see  Table  1).  Be¬ 
cause  optical  disks  are  only  touched  by  the 
light  beam,  they  are  very  safe;  vendors  guar¬ 
antee  their  media  readability  for  about  25 
years. 
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analog  form  on  paper.  The  paper  is  stored  in 
file  folders  contained  in  filing  cabinets. 
When  we  need  to  verify,  validate,  resolve  a 
dispute,  or  whatever,  in  all  likelihood  we 
must  physically  walk  to  the  files,  retrieve  the 
paper  file,  return  to  our  work  sites,  and  con¬ 
tinue  with  the  work  activity.  Often  retriev¬ 
ing  these  documents  is  a  frustrating 
experience.  Files  are  lost  or  misfiled,  they 
may  be  in  use  by  a  fellow  worker,  and  the 
process  of  finding  and  retrieving  filed  docu¬ 
ments  consumes  too  much  time. 


DP  Environment 


Paper  Environment 


DBMS 
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Fl|$urc  3.  Traditional  Papcr-BaMHl  Inforniatlon 
Manaf^ment 


Most  imaging  systems  use  WORM  optical 
disks  (rather  than  rewritable)  largely  because 
WORM  disks  cannot  be  altered.  For  exam¬ 
ple,  the  U.S.  Nuclear  Regulatory  Commis¬ 
sion  (NRC)  issued  a  guideline  stating:  "The 
optical  disk  technology  must  not  allow  dele¬ 
tion  or  modification  of  record  images"  (2). 
Since  WORM  disks  cannot  be  erased  and  re¬ 
written  they  cannot  be  deleted  or  modified. 

Information  Management  ~  Paper  versus 
Electronic 

Over  time,  information  management  in  the 
typical  office  has  taken  on  the  perspective 
depicted  in  Figure  3.  Office  workers  use  the 
computer  in  the  performance  of  their  daily 
work  routines;  but  for  the  most  part,  they  re¬ 
cord  or  document  the  results  of  their  work  in 


Under  imaging,  the  pattern  of  the  work  flow 
is  altered  dramatically  (see  Figure  4).  In¬ 
stead  of  storing  the  results  of  our  work  in 
analog  form  on  paper  we  store  and  manage 
the  information  in  electronic  form  on  optical 
disks.  These  "electronic  information  files" 
may  contain  both  image-based  information 
in  raster  bit-mapped  form  and  character  in¬ 
formation  that  we  produce  on  our  computer 
workstations  in  an  integrated  imaging  sys¬ 
tem.  Now  when  we  need  to  verify,  validate, 
resolve  a  dispute,  or  whatever,  we  no  longer 
must  physically  walk  to  the  files  to  retrieve 
the  paper  file.  Instead,  we  simply  retrieve 
and  display  the  images  of  the  document  on 
our  "image  workstation"  and  continue  our 
work  activity  without  interruption. 
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DP  Environment 


Figure  4.  Optical  Disk  Electrouic  Document  Image 
System  Information  Management. 


Converting  Paper  Documents  to  an  Imag¬ 
ing  System 

Document  conversion  is  an  expensive  com¬ 
ponent  of  any  major  imaging  implementa¬ 
tion;  often  it  is  at  least  as  costly  and  perhaps 
several  times  more  costly  than  the  system 
hardware  and  software  costs.  The  steps  in 
scanning  and  indexing  documents  include: 
document  preparation,  the  actual  scanning 
itself,  indexing,  quality  assurance,  and  refil¬ 
ing  or  destroying  the  original  documents. 
Many  imaging  systems  implementors  use 
service  bureaus,  companies  that  specialize  in 
the  conversion  process. 

Converting  analog  information  on  paper  to 
electronic  bit-mapped  information  in  an 
imaging  system  and  storing  these  "electronic 
document  images"  on  optical  disk  for  subse¬ 
quent  access  and  use  is  depicted  in  Figure  5. 

Document  preparation  can  be  a  critical  com¬ 
ponent  of  document  conversion.  It  is  much 
more  than  merely  removing  paper  clips,  sta¬ 
ples,  and  repairing  tom  and  damaged  pages; 
it  should  also  ensure  the  validity  of  the  docu¬ 
ment  base.  For  example,  it  should  address 
duplicate  copies  of  documents  in  the  files 
and  other  records  management  issues  that 
tend  to  corrupt  files.  An  imaging  system 


does  not,  by  itself,  ensure  that  a  mismanaged 
filing  system  suddenly  becomes  perfect. 

Quality  assurance  (QA)  is  another  important 
step  in  the  document  capture  process.  QA  is 
nothing  more  than  ensuring  that  the  bit¬ 
mapped  image  is  clear  and  legible  and  that 
all  of  the  information  is  included  in  the  bit 
map.  It  may  be  important  to  ensure  that 
there  is  no  skewing  of  the  images  or  that 
there  are  no  drop-out  colors.  This  step  re¬ 
quires  an  operator  to  visually  inspect  the 
document  images  and  consequently  is  a  te¬ 
dious  and  time-consuming  task.  However, 
as  operators  become  more  and  more  profi¬ 
cient,  the  QA  step  can  be  performed  on  a 
random  sampling  basis. 


The  Index  Database 

Note  that  Figure  5  introduces  another  aspect 
of  managing  document  information  in  image 
form,  namely  indexing  (see  upper  right  hand 
comer).  Bit-mapped  images  are  nothing 
more  to  the  computer  than  millions  of  pic¬ 
ture  elements  or  ones  and  zeros.  The  com¬ 
puter  does  not  recognize  this  bit-map  like  it 
does  character  information. 


Therefore,  all  imaging  systems  employ  an 
index  database  management  system 
(DBMS),  which  contains  the  essence  of  the 


Figure  S.  Steps  in  Convertlne  and  Using  Paper  Docnincnts  I 
in  an  Optical  Disk  Doenment  Imaging  System.  I 

^  J 
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information  written  on  the  original  paper 
document  page;  it  is  used  to  file  and  retrieve 
image  documents.  The  index  database  is  the 
management  heart  of  an  imaging  system;  it 
provides  the  only  road  map  to  find  informa¬ 
tion  that  has  been  scanned.  This  index 
DBMS  may  be  as  simple  as  a  single  entry; 
for  example,  a  case  number,  social  security 
number  or  contract  number.  On  the  other 
hand,  it  may  be  quite  complex;  for  example, 
a  multiple  field  DBMS  that  provides  many 
specific  elements  of  information  about  the 
document  that  has  been  scanned. 

In  fact,  the  index  DBMS  could  even  include 
the  complete  text  of  all  of  the  scanned  pages 
through  the  use  of  optical  character  recogni¬ 
tion  (OCR)  technology.  OCR  is  software 
which  is  able  to  take  a  bit-map  image  that 
represents  text  (such  as  from  a  scanned 
image  of  a  typed  page)  and  convert  it  to 
ASCII  text  characters  by  recognizing  pat¬ 
terns  in  the  image.  The  ASCII  characters  are 
then  used  to  index  the  bit-mapped  images. 
OCR  is  also  used  to  extract  information  from 
fields  or  zones  on  the  scanned  pages  such  as 
data  elements  from  a  form  or  other  standard 
document.  The  index  database  is  then  pop¬ 
ulated  from  the  OCR  output  and  either  a  full- 
text  index  or  fielded  database  indexing 
scheme  is  employed.  These  OCR  and  index¬ 
ing  schemes  can  be  very  effective,  but  they 
are  not  inexpensive  or  perfect. 

A  bar-code  can  also  be  used  to  provide  the 
crucial  or  key  identifying  information  ele¬ 
ment  in  a  document.  One  example  of  this  is 
computer  generated  delivery  orders  which 
must  be  scanned  to  retain  evidence  of  receipt 
of  a  delivered  item.  Placing  a  bar-code  on 
the  computer-generated  delivery  ticket  pro¬ 
vides  a  means  to  automatically  capture  the 
invoice  number  later  on  when  the  delivery 
receipt  is  scanned  into  an  imaging  system 
and  to  use  this  as  the  index  DBMS  entry. 


Optical  Disk  Electronic  Document  Image 
Management  Systems  Classifications 

Although  imaging  systems  are  becoming 
more  and  more  solution  specific,  some  effort 
has  been  devoted  to  classifying  and  tracking 
these  systems.  One  of  these  classification 
frameworks  divides  optical  disk  imaging  sys¬ 
tems  into  five  categories  (see  Table  2).  Note 
that  the  general  size  of  the  configuration  and 
its  costs  are  also  presented. 

/  \ 


TaUe  2.  Optical  Disk  Electronic  ImaipiiK 
Oassificalion  KVamrwork. 


SYSTEM 

USERS 

(#) 

IMAGES 

m 

COST* 

($) 

Single  User 

1-2 

50K 

30K-50K 

Work  Group 

3-10 

1M 

75K-300K 

Departmental 

11-50 

5M 

300K-750K 

Business  Unit 

51-100 

10M 

750K-2M 

Enterprise 

100++ 

10M+ 

2M-10M+ 

•  Excluding  Convorslon 


For  large  document  storage  and  retrieval  re¬ 
quirements,  optical  disk  systems  employ  a 
jukebox.  A  jukebox  is  a  device  that  provides 
storage  for  from  five  to  several  hundred  opti¬ 
cal  disks,  one  or  more  drives,  and  a  robotics 
mechanism  to  move  a  disk  from  the  storage 
area  into  the  drive  (see  Reference  1,  Figure 
5).  The  Sony(tm)  50-disk  jukebox,  for  ex¬ 
ample,  which  occupies  a  floor  space  of  a 
square  meter  or  less,  can  provide  storage  for 
327GB  (or  more  than  6  million  pages  of  pa¬ 
per).  Jukebox  storage  is  often  referred  to  as 
"near  on-line  storage;"  meaning  that  for  re¬ 
trieval  of  information  that  is  on  an  optical 
disk  platter  in  a  storage  slot,  the  jukebox  ro¬ 
bot  has  to  change  optical  disks  before  the  in¬ 
formation  can  be  accessed.  This  type  of 
retrieval  is  usually  accomplished  in  20  sec¬ 
onds  or  less  versus  one  or  two  seconds  for 
information  on  an  optical  disk  that  is  already 
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spinning  in  a  drive.  Large  document  imag¬ 
ing  configurations  with  a  jukebox  will  be 
presented  in  the  imaging  and  contracting 
section. 


Work  Flow 

Workflow  management  in  electronic  imag¬ 
ing  systems  is  the  ability  to  route  and  track 
electronic  documents  throughout  an  orga¬ 
nization  in  a  procedural  fashion  and  to  per¬ 
form  work  fiinctions  and  add  value  to  a 
document  electronically.  While  storage  and 
retrieval  is  usually  the  enticement  for  a  po¬ 
tential  user  to  consider  imaging,  it  is  work- 
flow  that  can  really  provide  the  most  benefit 
from  the  implementation.  Workflow  will 
also  be  discussed  in  the  contracting  example. 

IMAGING  AND  CONTRACTING 
Introduction 

The  use  of  an  optical  disk  electronic  imaging 
system  in  the  contracting  environment  has 
been  presented  in  this  forum  previously  (3). 
In  the  following  discussion  the  optical  disk 
imaging  concepts  described  above  are  inte¬ 
grated  into  a  real-world  federal  contracting 
function  (however,  the  identity  of  the  office 
is  not  disclosed). 

The  contracting  office  embodies  more  than 
100  contract  specialists  who  together  pro¬ 
duce  and  manage  about  1.5  million  pages  of 
contract  file  documents  annually.  Although 
automation  of  the  contract  production  pro¬ 
cess  is  presently  being  employed,  the  storage 
and  management  of  the  finished  contract 
document  are  still  being  conducted  with  pa¬ 
per  documents.  This  has  lead  to  the  twelve 
problem  areas  identified  in  Table  3. 


TaUe  3.  Contract  Offlor  ProUem  Afcat 

■  Lost  A  WnfitodOocumonla  •Filing  and  Working  Spoc* 

•  Contiad  FU*  Rotrioval  ■  Inlerfaca  With  Automatad 

•  Racordt  Managamani  Systama 

•RaproductioA  artd  •Cotlomar  Statua  Daak 

Diaaainination  •Eacaaaiva  Amount  Papar 

•  Fling  and  Ratriaving  Signad  •  Papariaaa  Offica  Goal 

Racaipt  Oocumania  aOparmtionalEffactivanaaa 

■Fling  and  Ratriaving  arnl  Efficiancy 

Computar  Status  Rapta 


V 

Contract  file  document  flow  is  depicted  in 
Figure  6.  Because  of  continuing  file  access, 
retrieval,  and  lost  files  problems,  contract 
files  are  maintained  primarily  at  the  special¬ 
ist's  work  site  exacerbating  the  filing  and 
working  space  problem.  Those  files  that  are 
maintained  in  the  centralized  contract  files 
experience  a  significant  frequency  of  lost, 
misfiled  and  out-of-file  conditions. 


Imaging  Alternatives  and  Impacts  on  the 
Problems 

How  can  imaging  help  alleviate  the  situa¬ 
tion?  Two  alternative  configurations  are 
proposed:  1)  a  self-contained  or  standalone 
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imaging  system  in  the  central  files  area  to 
house  the  central  files,  and  2)  an  office-wide 
integrated  optical  disk  imaging  system.  Of 
course,  numerous  configurations  somewhere 
between  these  two  extremities  are  also  possi¬ 
ble  and  indeed  may  offer  the  best  practical 
solution.  However,  due  to  the  space  con¬ 
straints  of  this  article,  they  are  not  discussed 
here. 


FAX 


Figure  7.  Configuration  Alternative  I  ••  Standalone  Imaging  S>*stem  for 
Storing  and  Retneving  Contract  Files. 


Standalone  Imaging  System  for  the 
Central  Files 

The  standalone  imaging  system  in  the  central 
files  area  to  house  the  central  files  offers  a 
partial  solution  to  this  contracting  office's 
difficulties.  This  configuration  (see  Figure 
7)  consists  of  the  following:  a  UNIX  image 
and  index  DBMS  server(s);  a  jukebox  pro¬ 
viding  two  optical  disk  drives  and  storage  for 
at  least  30  twelve-inch  dual-sided 
WORM  optical  media;  a  scan  station 
for  converting  the  hard  copy  paper 
documents  to  electronic  images;  two 
image-enabled  PC  workstations  for  in¬ 
dexing,  quality  assurance,  and  file  re¬ 
trieval;  a  high-speed  laser  printer;  and  a 
facsimile  server  for  FAX  input  and  out¬ 
put.  (Since  image  management  sys¬ 
tems  use  facsimile  compression 
standards,  a  standard  office  facsimile 
can  function  as  both  a  source  or  input 
station  and  as  an  output  station.) 

The  impact  of  this  Alternative  1  or 
standalone  configuration  on  office  con¬ 
tract  file  workflow  is  limited  (see 
Figure  8).  This  configuration  is  very 
good  for  meeting  the  general  contract 
file  storage  and  retrieval  requirements 
in  the  central  filing  area;  however,  it 
does  very  little  to  address  some  of  the 
office's  major  problems  (see  column 
labeled  "Alt.  1"  in  Table  4).  For  those 
problem  areas  that  relate  to  overall  or¬ 
ganizational  problems  and  goals,  the 
Alternative  1  configuration  also  falls 
far  short  of  the  office's  requirements. 

An  Office-wide  Integrated  Optical 
Disk  Imaging  System 


^  \ 


COPY  sot 
OOCl»«MIS 


JUKEBOX 


Figure  8.  Cootracl  File  Document  Flow  With  Altcmaiivc  I—  Stand 
Akme  Imaging  System. 
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Alternative  2  is  an  integrated  office-wide  op¬ 
tical  disk  document  imaging  system  (see  Fig¬ 
ure  9).  This  configuration  maintains  the 


Table  4,  Impact  of  Alternative  Configurations  and 
Workflow  on  Problems. 


A 


ALT  1 

ALT  2 

WF 

Lost  Documents 

M 

VH 

VH 

File  Retrieval 

M 

VH 

VH 

Records  Management 

L 

H 

VH 

Repro  &  Oissemin. 

L 

H 

H 

Sigrted  Receipt  Docs 

N 

VH 

VH 

Computer  Status  Rept 

N 

VH 

VH 

Work  Space 

N 

VH 

VH 

Automated  Sys. 

L 

M 

VH 

States  Desk 

L 

H 

VH 

Excess  Paper 

L 

H 

VH 

‘Paperless  Office* 

VL 

M 

VH 

Efficiency  &  Effect 

VL 

_ 

VH  1 

N :  None,  V\.  =  Very  Low;  L  :  Lcnv;  M  =  Ih4e<fivn  or  Moderate;  H  ;  ^h, 
VH  =  \^ry  Hii^  n  Highest 


UNIX-based  image  server  in  the  central  re¬ 
cords  center  but  moves  the  index  DBMS  to 
the  office's  existing  UNIX  computer  and 
connects  the  two  with  a  Local  Area  Network. 
All  of  the  features  of  the  imaging  system  in 
the  central  records  area  are  retained  in  this 
configuration  so  all  of  the  central  records 
functions  from  Alternative  1  are  retained  in 
this  alternative.  However,  the  jukebox  ca¬ 
pacity  and  the  number  of  optical  disk  platters 
must  be  increased  to  accommodate  the  addi¬ 
tional  active  contract  files. 


UNIX  DOCUMENT 
IMAGE  SERVER 


In  the  Alternative  2  configuration,  all  ap¬ 
propriate  office  staff  (Contract  Specialists, 
Contract  Officers,  supervisors,  office  man¬ 
agement,  oversight  and  policy,  contract 
closeout,  etc.)  are  provided  an  image-enabled 
PC.  An  image-enabled  PC  is  such  as  the  one 
shown  in  Figure  1  and  includes  a  19-inch 
high-resolution  high-quality  monitor  and  ap¬ 
propriate  hardware  and  software  that  enable 
it  to  simultaneously  access  images  from  the 
documc.it  imaging  system  on  optical  disk 
and  all  automated  systems  and  office  auto¬ 
mation  capabilities  of  the  existing  UNIX- 
based  Automated  Systems  computer. 

The  impact  of  Alternative  2  on  contract  file 
work  flow  is  very  striking  (see  Figure  10). 
Since  it  places  both  imaging  and  ADP  power 
in  the  hands  of  the  specialists  and  others  who 
create  contract  file  documents.  Alternative  2 
essentially  eliminates  all  paper  contract  file 
documents.  The  Alternative  2  contract  file  is 
a  composite  of  character  based  information 
from  the  automated  systems  and  of  scanned 
paper  supporting  documents  as  may  be  need¬ 
ed  from  the  imaging  system.  In  this  configu¬ 
ration,  the  electronic  contract  files  are 

_ captured  as  they  are  created  by 

their  creators,  the  specialists. 
Because  of  this.  Alternative  2 
has  a  high  to  very  high  impact 
on  most  of  the  office’s  problem 
areas  (see  column  "Alt.  2"  in 
Table  4). 


CENTRALIZED 
ELECTRONIC  FILES 
SYSTEM  MANAGEMENT 


CONTRACT  SPECIALISTS 
WORK  GROUPS 


INDEX 

DBMS 


CLOSEOUT,  POLICY, 
STATUS.  FSVIEW. 
MANAGEMENT,  ETC 


Figure  9.  Configuration  Alternative  2 
Di.tk  Imaging  .Sy,slem, 


OfTite-WiJe  Integrated  Optical 


Imaging 

Workflow 


and  Electronic 


Electronic  workflow  manage¬ 
ment  is  often  implemented  with 
optical  disk  imaging  systems. 
In  this  context,  workflow  is  the 
ability  to  route  and  track  elec¬ 
tronic  documents  throughout  an 
organization  in  a  procedural 
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ing  to  effectiveness  and  efficiency. 
A  schematic  representation  of  how 
electronic  workflow  might  work  in 
the  contracting  office  is  illustrated  in 
Figure  11.  Contract  specialists  (top 
of  Figure  11)  create,  modify  and 
maintain  the  contract  file  documents 
electronically,  which  are  housed  on 
the  optical  disk  imaging  system. 
Progressing  clockwise  in  Figure  11, 
once  the  contract  file  and  its  separate 
documents  have  been  created  and 
placed  on  the  optical  disk,  they  can 
be  reviewed  and  acted  upon  by  Con¬ 
tract  Officers  and  Managers 
electronically.  The  electronic  work- 
flow  software  manages  the  process 
of  routing  and  approving  contract 
documents  by  automatically  placing 


fashion,  and  to  perform  work  functions  and 
add  value  to  a  document  electronically. 
Workflow  is  a  relatively  low-cost  software 
addition  to  an  imaging  system. 


documents  in  "electronic  in-boxes." 
There  is  no  "float  time"  associated  with  this 
process  since  documents  are  not  managed  in 
paper  form. 


Adding  workflow  to  the  Alternative  2  con¬ 
figuration  for  contract  file  storage  and  re¬ 
trieval  will  help  oiu"  contracting  office  better 
address  its  problems,  especially  those  relat- 
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Continuing  clockwise  in  Figure  1 1 ,  Manage¬ 
ment  and  Oversight  can  electronically  review 
the  file  for  policy  and  compliance.  Contract 
closeout  can  not  only  access  the  file  electron- 

_  ically  but  the  workflow  software 

^  system  can  actually  schedule  the 
1  closeouts. 
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Figure  II.  Schematic  R^resentation  of  Contract  File  Creation  and 
Management  Under  a  Ofnce-wide  Imaging  System  With  Workflow, 


The  central  records  center,  where 
the  imaging  system  and  the  opti¬ 
cal  disks  reside  in  this  plan,  are 
able  to  use  the  imaging  system  to 
automate  contract  file  disposition 
and  removal  to  archives.  The 
workflow  software  also  provides 
a  much  greater  information  re¬ 
source  base  for  answering  cus¬ 
tomer  and  vendor  inquiries. 
Finally,  workflow  software  pro¬ 
vides  office  management  with 
access  to  not  only  the  contract 
files  and  their  component 
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documents  but  also  can  provide  statistical 
information  on  contract  actions  and  staff 
work  activities. 

Electronic  workflow  can  only  be  implem¬ 
ented  on  a  configuration  like  Alternative  2 
because  to  be  effective  all  office  members 
must  have  intimate  and  rapid  access  to  both 
images  and  ADP  functions.  Adding  elec¬ 
tronic  workflow  to  the  Alternative  2  configu¬ 
ration  has  the  potential  to  meet  all  of  the 
office's  needs  and  goals  (see  Table  4,  "WK" 
colurrm).  However,  in  order  to  implement 
workflow  serious  consideration  should  be 
given  to  process  redesign  or  reengineering  to 
eliminate  unnecessary  paper  based  operation¬ 
al  steps  and  to  take  advantage  of  the  full  po¬ 
tential  of  an  optical  disk  electronic  imaging 
system. 


SUMMARY 

Optical  disk  electronic  imaging  systems  can 
be  effectively  utilized  in  the  contract  office 
of  the  future.  What  strategy  should  the  con¬ 
tracting  office  follow  when  they  are  consid¬ 
ering  implementing  an  imaging  system? 

Imaging  systems  should  be  integrated  into  an 
organization's  overall  ADP,  office  automa¬ 
tion,  and  information  management  systems 
to  be  most  effective.  Simple  storage  and  re¬ 
trieval  imaging  applications  only  utilize  a 
small  fraction  of  imaging's  true  potential  in 
the  contracting  office. 

The  first  thing  to  do  is  to  become  imaging 
smart.  Attend  imaging  conferences  or  obtain 
the  services  of  a  qualified  imaging  consul¬ 
tant.  The  Association  for  Information  and 
Image  Management  maintains  a  large  library 
of  resources  that  can  assist  you  in  this 
endeavor. 


Let  business  and  work  considerations  drive 
your  imaging  decision.  Do  not  be  swayed  by 
vendors  or  others  who  are  pushing  a  product. 

Involve  your  staff  in  the  process.  Create  an 
Imaging  Task  Force  or  utilize  a  Joint  Appli¬ 
cation  Design  procedure.  Involve  specialists, 
records  managers,  managers,  and  informa¬ 
tion  systems  people  in  the  process.  Since 
work  procedures  changes  are  likely,  an 
educational  effort  is  necessary  to  sell  the  new 
method  of  doing  work. 

Be  certain  to  create  a  flexible  system,  one 
that  is  capable  of  adapting  to  change  and 
new  applications.  There  are  numerous  ex¬ 
amples  of  imaging  systems  implementations 
that  are  not  being  used  because  they  were  not 
capable  of  adapting  to  new  environments  and 
to  new  and  additional  requirements. 

Finally,  implement  work  process  changes  to 
take  full  advantage  of  the  potential  of 
imaging. 
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Charles  R.  Thompson,  CPCM 


ABSTRACT 

The  first  ten  years  of  the  Small  Business 
Innovative  Research  (SBIR)  program  have 
shown  the  program  to  be  quite  valuable  and 
successful.  Both  government  and  the  private 
sector  have  received  significant  benefits  to 
date.  However,  there  is  an  untapped 
resource  in  the  SBIR  program:  innovative 
proposals  that  have  significant  technical  merit 
and  commercial  application  potential  that  go 
unfunded  by  agencies  each  year.  Agencies' 
inability  to  pursue  them  may  result  from 
insufficient  SBIR  funding,  or  because  they 
are  not  sufficiently  relevant  to  the  agencies' 
missions. 

Our  hypothesis  presented  herein  is  that  some 
number  of  those  meritorious  but  unfunded 
proposals  could  be  of  interest  and  value  to 
prime  contractors,  academic  institutions  and 
other  possible  investment  capital  sources,  and 
might  be  picked  up  for  support  if  their 
existence  and  merits  could  be  made  known 
widely.  The  paper  suggests  a  concept 
referred  to  as  "A  Marketplace  of  Ideas"  to 
help  facilitate  pursuit  of  some  of  these 
innovative  concepts  by  interested  parties 
outside  the  government,  in  the  belief  that 
such  activities  would  be  in  the  national 
interests.  The  Marketplace  of  Ideas  would 
provide  all  SBIR  agencies  the  opportunity  to 
publicize  all  suitable  unfunded  Phase  I  and 
Phase  II  proposals  through  an  on-line 
database  of  proposal  abstracts  open  to  the 
public  with  relatively  little  expense.  * 

INTRODUCTION 

Public  Law  97-219  enacted  1982,  created  the 
Small  Business  Innovative  Research  (SBIR) 
program.  Beginning  in  Fiscal  Year  1983,  it 
required  all  Federal  agencies  with  annual 
extramural  R&D  budgets  in  excess  of  $100 
million  to  set  aside  a  portion  of  t.'^ir  budgets 
to  fund  small,  high-tech  busine  ses  for  the 
development  of  innovations  they  have 
proposed.  In  recent  years,  the  raction  has 
been  1.25  percent. 


*  The  assistance  of  Harry  W.  Johnson,  Director,  SBIR 
Program.  NASA,  is  acknowledged  and  appreciated.  This 
article  reflects  only  the  author's  views  and  does  not 
represent  the  position  of  NASA  or  any  other  agency. 


The  enabling  legislation  stipulated  four 
purposes  of  SBIR: 

To  stimulate  technological  innovation; 

To  use  small  business  to  meet  Federal  R&D 
needs; 

To  increase  private  sector  commercialization 
of  innovations  derived  from  Federal  R&D; 
and. 

To  foster  and  encourage  minority  and 
disadvantaged  participation  in  technological 
innovation  (SBIR  is  not  a  minority  or 
socioeconomic  disadvantaged  set-aside, 
however). 

Eleven  agencies  conduct  SBIR  programs  and 
have  applied  approximately  $  2.75  billion  to 
the  program  from  inception  through  Fiscal 
Year  1991. 

The  program  has  been  judged  quite 
successful  both  by  the  government  and  the 
private  sector.  Agencies  with  SBIR 
programs  have  availed  themselves  of  some  of 
the  most  original  thinking  in  the  nation  on  a 
wide  variety  of  topics  that  relate  to  their 
mission  interests  and  objectives,  and  many 
offerors  compete  vigorously  for  grants  or 
contracts.  The  program  has  been  re¬ 
authorized  twice  by  subsequent  legislation,  in 
1986,  and  in  1992,  with  each  re-authorization 
broadening  and  extending  the  program.  The 
current  sunset  date  for  the  program  is  October 
1,  2000,  but  further  extension  will  be 
considered  by  the  Congress  before  that  date. 

A  measure  of  the  perceived  program  value 
and  success  is  that  the  October  1992 
legislation.  Public  Law  102-564,  increased 
agency  SBIR  set-asides  to  1.5%  of  their 
extramural  budgets  starting  in  1993,  to  2.0% 
in  1995.  and  to  2.5%  in  1997.  The  1992 
legislation  also  increases  emphasis  on  the 
goal  of  commercial  applications  in  the  private 
sector,  and  charges  participating  agencies 
with  finding  means  to  encourage  and  facilitate 
this  objective. 

SBIR  is  conducted  in  three  pha.ses.  Phase  I 
explores  the  feasibility  of  selected 
innovations  through  6-month  grants  or 
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contracts,  normally  not  exceeding  $50,000. 
Phase  II,  which  is  restricted  to  Phase  I 
participants,  is  the  principal  SBIR  activity.  It 
continues  the  development  for  up  to  two 
years  of  (typically)  one-third  to  one-half  of 
the  Phase  I  projects,  using  contracts  normally 
not  exceeding  $500,000.  Both  Phase  I  and 
Phase  II  projects  within  each  agency  are 
chosen  competitively  on  the  basis  of 
evaluations  of  proposals  received  in  response 
to  program  solicitations  issued  by  the  agency, 
suitability  for  agency  programs  and  missions, 
and  available  funding. 

Phase  ill  is  the  development  of  commercial 
applications  by  the  small  business 
contractors,  but  it  can  also  be  the  extension  of 
project  development  and/or  end-use 
procurement  by  the  government  of  projects 
carried  through  Phase  II.  Phase  III  is  not 
supported  by  the  SBIR  funding  set-aside,  but 
must  use  private  sector  funds  in  the  case  of 
commercial  applications,  and  non-SBIR 
federal  funds  in  the  case  of  continued  agency 
development  support. 

The  large  majority  of  innovations  proposed 
by  small  businesses  for  SBIR  support  are 
never  funded  by  the  agencies.  While  a  large 
fraction  may  be  judged  unsuitable  for  award 
selection,  many  proposals  which  have 
considerable  merit  are  not  selected  due  to 
funding  limitations  despite  their  technical 
interest  and  merit.  Any  of  those  worthy  ideas 
that  are  not  explored  are  costly.  Those  not 
selected  entail  a  loss  of  resources  expended 
by  the  proposing  firms  in  developing  their 
proposals,  but  even  more  significantly,  they 
may  represent  potentially  much  larger  losses 
to  the  nation  due  to  the  lost  opponunities  for 
ultimate  applications  of  the  innovations. 

This  paper  presents  a  concept  called  "A 
Marketplace  of  Ideas"  for  the  SBIR  program 
in  which  agencies  would  assist  small 
businesses  seeking  support  for  their 
innovative  ideas  that  the  government  cannot 
fund  through  the  SBIR  or  other  agency 
programs,  the  concept  could  help  prevent  the 
loss  of  valuable  development  opportunities 
by  assisting  prime  contractor,  and  other 
possible  private  sector  funding  sources  (and 
possibly  nonprofit  sources  as  wjU)  to  learn 
about  those  small  businesses  and  their 
innovative  ideas.  The  potential  l>*nefits  to  the 


Nation  and  its  competitive  strength  could  be 
very  great. 

DISCUSSION 

Since  the  beginning  of  the  SBIR  program  in 
1983,  it  is  estimated  that  more  than  5,000 
small  businesses  have  submitted  at  least 
140,000  Phase  I  proposals  through  Fiscal 
Year  1991,  to  the  government  proposing 
innovations  in  response  to  the  annual 
Program  Solicitations  published  by  each 
participating  agency.  There  have  been  about 
16,00()  Phase  I  awards  and  5,100  Phase  II 
awards  through  the  end  of  Fiscal  1991 
government-wide,  which  means 
approximately  one  in  eight  Phase  I  proposals 
received  are  funded,  and  only  about  one-third 
to  one-half  of  the  funded  Phase  I  projects  are 
subsequently  selected  for  Phase  II 
continuation.  The  table  below  illustrates 
recent  statistics  from  the  1990  SBIR  program 
of  the  Department  of  Defense,  whose  SBIR 
set-aside  annually  accounts  for  more  than  half 
of  the  total  SBIR  funding  of  all  agencies 
combined. 


Service 

Proposals 

Phase  I 

% 

Reed. 

Awards 

Awarded 

Army 

998 

92 

9.2 

Navy 

2139 

323 

15.1 

AF 

3479 

337 

9.7 

DARPA 

596 

100 

16.8 

DNA 

213 

14 

6.5 

SDIO 

860 

155 

18.0 

Total 

8385 

1021 

12.2% 

The  table  shows  that  DOD  awarded  Phase  1 
contracts  for  12.2%  of  the  proposals 
received.  Considering  the  program  guidance, 
fund  availability,  award  selection  criteria  and 
other  factors,  perhaps  the  12.2%  award  rate 
is  the  best  obtainable  result. 

However,  the  table  also  indicates  that  87.8% 
of  the  proposals  were  not  awarded.  Of  those 
7,360  proposals  not  awarded,  we  speculate 
that  some  are  not  worthy  of  award  on 
technical  grounds.  In  addition,  NASA  SBIR 
program  officials  similarly  indicate  that  many 
of  the  Phase  1  proposals  it  reviews  are  not 
viable  and  should  not  be  funded  for  technical 
reasons.  However,  we  believe  that  there  may 
be  merit  in  some  of  the  proposals  not 
awarded,  and  that  they  were  not  awarded 
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simply  because  of  funding  or  other  program 
limitation.  We  believe  that  some  of  the 
unfunded  proposals  are  a  resource  that  is 
going  untapped. 

Several  agencies  (NASA,  for  example) 
estimate  that  half  to  two-thirds  of  the  Phase  I 
proposals  they  receive  are  not  suitable  for 
award  by  them,  for  a  variety  of  reasons. 
Unsuitable  proposals  may  be  based  on 
erroneous  physical  principles,  or  may  be 
viewed  as  infeasible  or  impractical.  Some 
may  simply  be  off-target  in  not  meeting 
agency  needs.  Still  others  may  contain  good 
ideas  that  have  already  been  investigated. 
However,  several  agencies  estimate  that  the 
number  of  proposals  they  find  suitable  for 
award  selection  is  roughly  twice  the  number 
they  are  able  to  fund.  Our  hypothesis  is  that 
some  of  the  meritorious  innovations  agencies 
cannot  fund  may  represent  potentially 
significant  losses  to  the  country  if  not 
pursued  outside  the  SBIR  program. 

The  situation  in  Phase  II  is  similar,  but 
probably  of  even  greater  significc nee.  Phase 
II  proposals  concern  innovations  whose  basic 
feasibility  has  already  been  explored  during 
Phase  I.  Not  all  Phase  I  projects  may  be 
found  to  have  enough  value  to  warrant 
continued  funding  by  the  government,  but  as 
in  Phase  I  selections,  the  number  of  Phase  II 
projects  funded  each  year  is  usually  limited 
by  available  funds,  not  by  a  shortage  of  good 
proposals  from  which  to  choose.  Agencies 
typically  are  able  to  select  perhaps  only  half 
to  two-thirds  the  number  found  to  be  award¬ 
worthy.  However,  all  those  award-worthy 
proposals  represent  ideas  whose  feasibility 
has  been  demonstrated  in  Phase  I  and 
represent  significant  investments  on  the  part 
of  both  the  government  and  the  small 
businesses  that  did  the  research.  The 
likelihood  of  loss  to  the  nation  by  not 
continuing  development  of  these  ideas  could 
be  greater  than  for  many  of  the  apparently 
meritorious  Phase  I  concepts  that  were  not 
selected  for  award  and  whose  feasibility  was 
never  established. 

It  is  suggested  that  each  agency  should  make 
available  -  with  the  approval  of  the  proposing 
firm  -  the  abstract  of  technically  acceptable 
but  un-awarded  Phase  I  and  Phase  II 
proposals  to  industry  and  academia.  The 


primary  objectives  are  clear; 

*  To  assist  small  businesses  in  their 
marketing  activities  without  increasing  their 
administrative  and  marketing  costs,  by 
exposing  their  innovations  to  a  broad  range 
of  potential  sources  of  development  funds 
and  other  business  arrangements  for  pursuing 
development.  In  many  instances,  small  firms 
might  not  know  of  or  be  able  to  afford  such 
opportunities  on  their  own. 

*  To  make  available  to  industry  and 
academia  an  organized  display  of  innovations 
and  available  talent  from  which  they  can 
decide  whether  further  inquiries  may  be  in 
their  interests.  This  should  open  up 
otherwise  unknown  opportunities  for 
increasing  their  own  corporate  objectives 
through  new,  innovative  ideas. 

Agencies  having  SBIR  programs  would  be 
encouraged  to  provide  the  necessary  data 
from  their  annual  proposal  information 
databases,  indicating  which  proposals  have 
been  selected  for  award,  and  also  if  feasible 
to  do  so,  noting  those  proposals  judged  to 
have  sufficient  merit  within  their  agency  to 
have  been  selected  for  award  by  the  agency 
had  additional  funds  been  available. 
Proposers  would  also  need  to  give  assent  to 
the  exposure  of  the  abstract  and  title  data  of 
their  proposal.  Those  who  decline  would  not 
have  their  proposals  included  in  the  database. 

Other  potential  users  of  the  database  could  be 
the  many  state-supported  organizations 
encouraging  small  business  entrepreneurs 
and  supporting  local  and  regional  research 
and  development.  Many  states  have 
programs  complementary  to  the  federal  SBIR 
programs  involving  some  form  of  economic 
assistance  and  other  business  development 
support,  including  some  form  of  matching 
funds  for  firms  winning  SBIR  awards. 
Some  states  provide  direct  funding,  grants, 
loans,  loan  guarantees,  tax  advantages  or 
other  information  to  small  businesses.  A 
1987  study  by  the  National  Governor's 
Association  reported  that  there  were  31  states 
with  offices  that  promote  the  growth  of 
technology-based  busines.ses.  However,  few 
of  such  organizations  are  able  to  stay  abreast 
of  all  potential  sources  of  innovations  that 
could  be  of  interest  to  them.  The  proposed 
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database,  "Marketplace  of  Ideas",  may  be  of 
direct  interest  to  the  states  with  economic 
assistance  programs. 

The  Marketplace  of  Ideas  initiative  would 
provide  another  beneficial  opportunity  for 
Federal  agencies  themselves.  Both  the 
Administration  and  the  Congress  are  vitally 
concerned  with  technology  transfer  from 
Federal  R&D  programs  to  the  private  sector 
to  benefit  the  overall  economy  and  national 
well-being.  But,  heretofore,  it  has  been  a 
not-uncommon  perception  -  whether  true  or 
not  -  that  Federal  agency  efforts  in  that 
direction  may  be  less  than  enthusiastic  or 
effective.  The  Marketplace  of  Ideas  would  be 
one  positive  and  visible  indication  that  would 
help  agencies  dispel  possible  images  of 
reluctance  or  inability  to  advance  tech-transfer 
objectives. 

IMPLEMENTATION 

The  Marketplace  of  Ideas  concept  could  be 
implemented  with  litde  expense  or  additional 
effort  for  an  agency.  A  preliminary 
implementation  plan  should  consider  these 
features: 

Step  1  -  Survey 

A  selected  lead  agency  conducts  a  survey  to 
determine  potential  acceptability  of  this  idea 
by  gathering  comments  on  the  concept  of  an 
SBIR  'Marketplace  of  Ideas'  from: 

*  Small  firms  that  have  proposed  on,  but  not 
won,  contracts  in  the  SBIR  program  (Phase  I 
or  II)  in  the  past  two  years 

*  Potential  large  business/university  users  of 
such  a  database  of  proposals 

*  Trade  associations  and  other  interested 
parties  (NSIA,  NCMA,  etc.) 

Conducting  this  survey  and  summarizing  the 
results  in  a  report  and  briefing  for  the  lead 
agency,  the  Small  Business  Administration 
and  the  Congress,  are  suitable  tasks  for  a 
contract  to  a  small  firm.  The  effort  could 
possibly  be  accomplished  in  60-90  days  after 
contract. 


Step  2  -  Database  Establishment 

Assuming  an  affirmative  result  in  the  survey, 
the  lead  agency  awards  a  contract  to  a  data 
processing  firm  to 

*  Create  a  database  of  all  SBIR  Phase  I 
proposal  abstracts  that  were  technically 
acceptable  but  not  funded,  and  those  Phase  I 
projects  that  were  not  selected  for  further 
funding  in  Phase  II 

*  Obtain  authorization  from  the  small 
business  to  list  his  or  her  abstract  in  the 
database 

*  Vigorously  market  on-line  access  to  the 
database  by  the  public,  thereby  generating 
revenue  to  pay  for,  or  at  least  offset,  the  cost 
of  the  effort 

*  Report  revenue  and  expense  to  the 
contracting  officer 

*  Operate  the  on-line  database  for  at  least  a 
year  to  prove  the  concept 

*  Provide  for  periodic  assessment  of  the 
usefulness  of  the  database,  evaluating  user 
suggestions  for  improvement. 

Step  3  -  Database  Operation  and  Extension 

Assuming  success  of  the  survey  and 
demonstration  steps,  the  lead  agency 
modifies  its  SBIR  policy  to  permit  an 
operational  extension  of  the  database,  with 
the  intention  of  fully  developing  the  concept 
based  on  test  experience. 

The  agency  also  executes  a  public 
information  program  to  support  the  concept 
of  a  "Marketplace  of  Ideas"  and  to  publicize 
its  success,  to  include  Congressional 
notification  of  a  successful  initiative. 

Other  SBIR  agencies  then  adapt  the  lead 
agencies  implementation  to  their  own 
programs. 

The  SBIR  "Marketplace  of  Ideas"  should  be 
endorsed  by  every  agency  Director  of  Small 
and  Disadvantaged  Business  and  by  the 
Small  Business  Administration. 
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The  Marketplace  of  Ideas  could  well  be  an 
initiative  adopted  by  the  SEA  and  operated  on 
a  government-wide  basis  for  the  eleven 
agencies  that  now  participate  in  the  SBIR 
program. 


CONCLUSION 

The  Marketplace  of  Ideas  identifies  abstracts 
of  technically  acceptable  SBIR  Phase  I  and  II 
proposals  that  were  not  selected  for  funding, 
and  displays  them  in  a  computer  searchable, 
on-line  database  available  to  the  public.  The 
database  will  be  advertised  publicly  and 
especially  to  other  government  agencies, 
prime  contractors,  universities,  and  state 
government  economic  development  agencies. 
The  principal  benefit  of  making  available 
these  unawarded  SBIR  proposals  is  to  give 
them  a  greater  chance  of  obtaining  funding 
from  another  source  without  the 
administrative  expense  of  making  another 
proposal.  The  Marketplace  of  Ideas  can  be 
established  in  a  cost-effective  manner  with  a 
revenue-generating  feature  so  as  to  make  it  a 
no-cost  or  very  low  cost  investment  by  the 
agency.  This  idea  is  easily  adaptable  to  all 
federal  agencies  with  SBIR  programs. 

Author  Note 

Charles  R.  Thompson  is  a  forme*  contracting 
officer  with  the  US  Army.  He  is  a  senior 
staff  consultant  with  Consulting  On 
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ABSTRACT 

Electronic  data  interchange  (EDI), 
used  in  conjunction  with  current  bar  code 
technology,  offers  almost  unlimited  possi¬ 
bilities  for  improving  business  efficiency  in 
private  and  public  sectors  of  industry.  This 
article  documents  how  EDI  technology  has 
been  utilized  by  the  Naval  Air  Warfare  Center 
Weapons  Division  (NAWCWPNS),  China 
Lake,  Calif.,  to  improve  its  procurement,  fi¬ 
nancial,  and  supply  operations. 

In  January  1989,  Center  management 
chartered  a  study  team  to  investigate  the 
small-purchase  acquisition  process.  The 
team's  findings  confirmed  that  the  process 
was  bogged  down  with  paper  trails  and  man¬ 
ual  approvals  and  was  not  meeting  customer 
needs. 

Small  Procurement  Electronic  Data 
Interchange  (SPEDI)  was  bom  after  research¬ 
ing  government  and  private  sector  purchasing 
operations.  Using  an  EDI  systems  contract¬ 
ing  approach,  the  process  allows  customers 
to  place  electronic  orders  against  established 
contracts  utilizing  on-line  catalogs.  All  ap¬ 
provals,  including  funds  availability,  are  also 
accomplished  electronically.  Orders,  which 
take  the  form  of  bar-coded  labels,  are  printed 
at  the  contractor's  facility  and  are  affixed  to 
the  packaged  delivery  items.  The  contractor 
then  has  24  hours  to  deliver.  The  receiving 
process  is  accomplished  by  scanning  the  bar 
code  on  the  package  labels,  an  action  which 
also  creates  the  contractor's  invoice.  The  use 
of  SPEDI  has  reduced  payment  to  less  than 
10  days. 

The  NAWCWPNS  study  team  antici¬ 
pated  that  by  entering  into  longer  duration 
contracts  with  reduced  payment  terms, 
quantity  discounts  of  10%  could  be  obtained, 
ftesently,  two  contracts  are  on  line  with  dis¬ 
counts  exceeding  40%.  Performance  has 


been  excellent  and  is  tracked  by  the  SPEDI 
software,  as  is  historical  data  such  as  usage 
and  pricing. 


INTRODUCTION 

On  3  January  1989,  Naval  Weapons 
Center  (NWC — which  was  officially  desig¬ 
nated  NAWCWPNS  on  1  January  1992) 
management  chartered  a  team  to  investigate 
the  Center's  small-purchase  system  (actions 
of  $25,000  or  less);  identify,  describe,  and 
prioritize  actual  NWC  needs;  and  develop  a 
process  that  would  minimize  the  time  and  ef¬ 
fort  necessary  to  acquire  materials  and  sup¬ 
plies.  A  systems  viewpoint  was  advocated 
that  would  emphasize  technical  and  program 
needs  and  would  consider  the  requirements 
of  each  functional  area  affected  by  the  new 
process.  The  team  was  composed  of  four 
disciplines:  automation,  comptroller,  supply, 
and  procurement,  with  a  software  engineer  as 
the  team  leader.  The  final  report,  due  in  April 
1989,  was  to  include  an  analysis  of  the  exist¬ 
ing  small-purchase  system,  a  recommended 
new  approach,  anticipated  savings,  level  of 
in-house  effort,  and  costs  required  to  imple¬ 
ment  the  new  process. 

To  accomplish  its  task,  the  team  initi¬ 
ated  a  two-pronged  approach.  Each  of  the 
line  support  activities — procurement,  supply, 
and  comptroller — reviewed  their  processes 
associated  with  satisfying  small-purchase  re¬ 
quirements.  At  the  same  time,  the  team  began 
extensive  data-gathering  to  learn  how  other 
activities  in  both  the  government  and  the  pri¬ 
vate  sector  were  accomplishing  their  purchas¬ 
ing  needs.  In  conjunction  with  this  phase  of 
the  research,  a  formal  Request  for 
Information  (RFI)  was  issued  to  industry  to 
determine  what  technical  expertise  was  avail¬ 
able  to  assist  activities  in  a  process  improve¬ 
ment  endeavor  of  this  type. 
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FINDINGS  to  determine  if  it  could  be  purchased  locally 

from  the  open  market  or  if  some  mandatory 
The  small-purchase  process  analysis  source,  such  as  the  Federal  Prisons  or 
demonstrated  that  during  FY  88,  routine.  General  Services  Administration  (GSA),  was 
nonurgent  requirements  had  an  average  lead  directed, 
time  of  95  days  from  conception  to  NWC  de¬ 
livery,  and  an  average  cost  of  $154.08  for  After  approval  for  open-market  pro- 

each  stub  (purchase  request)  submitted.  The  curement,  the  requirement  was  compet^  to 

smaU-purchase  flow  process  at  that  time  is  obtain  the  best  price  for  the  government,  a 

shown  in  Figure  1.  purchase  order  was  issued  to  the  contractor, 

and  the  comptroller  was  notified  of  the  funds 
The  typical  purchase  request  required  obligation.  On  the  average,  the  screening  and 
the  customer  to  manually  research  the  item  of  procurement  processes  added  an  additioncd 
interest,  initiate  a  purchase  request  form,  ob-  30  days, 
tain  management  and  budget  approvals,  con¬ 
duct  special  reviews  to  determine  if  the  item  Routine  deliveries  were  averaging 

was  subject  to  any  federal,  state,  or  local  45  days  after  the  vendor  received  the  order, 

regulations  (environmental,  health,  etc.)  and  Receipt  at  NWC  consisted  of  inspection  and 

then  submit  the  request  to  the  procurement  acceptance  of  the  item  by  the  Supply 

office.  At  that  time  the  comptroller  was  noti-  Department  with  eventual  delivery  of  the  item 

fied  to  commit  funds  in  anticipation  of  the  to  the  customer.  A  receiving  report  was 

item's  procurement.  This  process  typically  submitted  to  the  comptroller  for  final  costing 

took  the  customer  20  days  to  accomplish.  of  the  obligation.  Payment  to  the  contractor 

then  followed  within  another  60  to  90  days. 

Prior  to  entering  the  procurement  of¬ 
fice,  the  request  had  to  be  screened  by  the  The  team's  findings  confirmed  that 

Technical  Branch  of  the  Supply  Department  the  small-procurement  purchasing  process 


FIGURE  1.  Small  Procurement  Flow  Process  (FY  88). 
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was  bogged  down  with  paper  trails  and  man¬ 
ual  approvals;  a  self-serving  system  that  was 
not  meeting  the  customers'  ne^s. 

The  RFI  produced  over  20  respon¬ 
dents  who  possessed  a  significant  degree  of 
expertise  in  the  area  of  procurement  process 
improvement  The  team  also  visited  several 
contractor  facilities  and  other  government  ac¬ 
tivities  to  see  how  they  were  performing  their 
procurement  functions.  It  became  immedi¬ 
ately  apparent  that  almost  every  successful 
process  viewed  involved  some  type  of  au¬ 
tomation  used  in  conjunction  with  EDI  and 
bar  code  technology.  Of  particular  interest 
were  the  systems  at  Sandia  National 
Laboratories  and  Los  Alamos  National 
Laboratories  in  New  Mexico.  They  had  in 
place  a  series  of  requirement-type  contracts 
for  commodities  ordered,  received,  and  paid 
for  using  EDI  and  bar  code  technologies.  The 
SPEDI  concept  was  patterned  after  portions 
of  these  examples. 

After  analyzing  the  accumulated  data 
and  comparing  the  results  to  the  Center's 
needs,  the  team  recommended  a  systems 
contracting  approach  that  would  take  full  ad¬ 
vantage  of  state-of-the-art  EDI  and  bar  code 
technologies.  The  approach  was  approved  by 
Center  management  in  September  1989.  Since 
the  small  procurement  arena  was  the  area  be¬ 
ing  explored,  SPEDI  was  bom. 


DISCUSSION 

The  SPEDI  process  is  implemented 
through  three  mechanisms: 

1.  Delegation  of  controlled,  limited 
small-procurement  authority  to  the  customers 

2.  The  award  of  long-term  systems 
contracts  for  commodity  classes  of  supplies 
and  materials 

3.  The  utilization  of  computer  tech¬ 
nology  to  provide  communication,  control, 
and  accountability 


The  first  two  mechanisms  of  this  pro¬ 
cess  could  be  implemented  only  through  au¬ 
tomation  and  by  themselves  could  not  pro¬ 
vide  the  integrated  benefits  of  a  total  systems 
approach.  However,  the  third  mechanism 
was  much  more  complicated. 

Communication  includes  replacing  the 
paper  procurement  request  with  an  electronic 
equivalent  that  provides  catalog  searches, 
procurement  path  determination,  order 
placement,  receiving,  payment  initiation,  and 
trouble  processing  through  automation. 

Control  consists  of  automated  pro¬ 
curement  authority  determination;  funds  vali¬ 
dation;  hazardous,  critical,  or  sensitive  mate¬ 
rial  monitoring;  arid  delivery  stage  tracking. 

Accountability  includes  automated 
tracking  of  purchases  by  delivery  stage,  shop 
code,  type  of  funds,  type  of  item,  buyer  his¬ 
tory,  and  vendor  delivery.  This  process  flow 
is  shown  in  Figure  2. 

Development  of  the  SPEDI  informa¬ 
tion  system  featured  a  relational  database 
configured  in  a  client-server  architecture. 
Access  to  the  client  is  achieved  from  each 
customer's  desk  through  a  networking  or 
modem  communication.  Customers  with 
authority  to  order  through  the  system  are 
called  SPEDI  Ordering  Officers  (SOOs). 
SOOs  use  SPEDI  to  search  electronic  cata¬ 
logs  and  place  automated  delivery  orders 
(DOs)  from  their  remote  workstations 
through  the  client  server  directly  to  systems 
contract  vendors  using  EDI. 

The  SOO  receives  a  warrant  from  the 
Head  of  the  Procurement  Office  after  viewing 
a  48-minute  video  and  receiving  hands-on 
training  in  the  database's  use.  Because  of  the 
extensive  checks  built  into  both  the  systems 
contracts  and  automation,  the  SOO's  author¬ 
ity  is  quite  limited  and  is  rigidly  controlled. 
This  extensive  control  and  accountability  al¬ 
lows  large  numbers  of  SOOs  to  be  estab¬ 
lished,  which  reveals  the  heart  of  the  pro¬ 
cess — allowing  the  customer  to  deal  directly 
with  the  contractor. 
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PROCUREMENT  AUTHORITIES 
MANAGEMENT  APPROVALS 
TECHNICAL  SCREENING  -f/j 
CATALOGUE  _  /// 


PURCHASE  REQUEST 
(IMMEDIATE) 


AUTOMATION 

- 1 - 'T - 

FUNDS  CHECKS 


EDI  DO(1  DAY) 


ALL  PROCUREMENT  REPORTS 
HAZARDOUS  MATERIAL  REPORTS 
VENDOR  PERFORMANCE  REPORTS 
AUDIT  REPORTS 


/ 

PAYMENT 


VENDOR 


COMPTROLLER 


24  HOURS 


RECP.  ACK. 


RECEIVING 


FIGURE  2.  SPEDI  Process  Flow. 


The  SOO's  authority  to  procure  spe¬ 
cific  items  (i.e.,  precious  metals,  chemicals, 


made.  Delivery  orders,  which  take  the  form 
of  bar-coded  labels,  are  printed  at  the 


etc.)  is  checked  electronically  at  the  time  the  vendor's  site  and  are  attached  to  the  delivery 
order  is  placed.  An  interface  with  the  package.  No  procurement  documentation 
accounting  system  ensures  that  the  funding  other  than  the  3-  by  4-inch  bar  code  label  is 


citation  is  valid,  contains  sufficient  funds, 
and  is  proper  for  the  type  of  purchase  being 


produced.  A  sample  delivery  order  is  shown 
in  Figure  3. 


SPE021 342001 32 


MR#:  SPE02-1342-00132 
Drop:  Bldg:  00060 

Partial:  N 

Min  Partlal: 

Hazardous: 

Plant  Property  (Y/N):Y 
Comments: 


RTE:  1-A 
Rm;  NONE 
Security  (Y/N):  N 

Substitute  (Y/N):  Y 
Sensitive  (Y/N):  N 


DESCRIPTION:  DISKETTE,  FLEXIBLE,  3.5"  SSDD,  FOR 
QTY:  1  U/l:  BX  Unit  Price;  $6.84 

Mig  Part#:  MF1  D  Mig  Code: 

Catalog  Item#;  S00000007 

EDI-Date  Time:  10-Dec-1991  14:37:22 

Contract#  :  N6053091F0012  M|o  Sen  Ten:  1199269GLF 

SOO/Ph:  CHERYL  VILLALBA  /  939-3180/3184 

REQR/Ph:  MARY  TEST  /  939-3486 


FIGURE  3.  SPEDI  Delivery  Order. 
(Actual  Size.) 
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The  vendor  has  a  limited  time  DETAILED  NARRATIVE 

frame  in  which  to  deliver;  typically,  at  DESCRIPTION  OF  THE  SYSTEM 

NAWCWPNS,  it  is  24  hours.  Upon 

receipt  of  an  order,  government  receiving  A  context  diagram  showing  the 

personnel  scan  the  bar  code  label.  This  complete  SPEDI  system  is  presented  in 

process  electronically  creates  receiving  and  Figure  4.  The  automation  is  accessed  by  the 

costing  documentation  and  initiates  SOO  who  logs  into  the  database  and  provides 

payment  to  the  vendor.  The  system  is  a  unique  user  identification  (password).  The 

designed  so  no  partial  deliveries  are  password  is  used  to  determine  authorities  that 

accepted.  If  the  entire  quantity  cannot  be  the  SOO  has  and  the  funds  that  he  or  she  may 

furnished,  the  shortage  is  canceled  and  use.  Upon  access  to  SPEDI,  the  SOO  has 

must  be  reordered.  Bar  code  scanning  is  three  basic  options:  place  or  change  an  order, 

also  used  to  indicate  when  the  item  is  perform  inquiries  (status  of  an  existing 

moved  from  the  dock  onto  a  delivery  tmck  order  or  to  obtain  data  on  a  series  of  past 

and  is  finally  delivered  to  the  customer,  orders),  and  search  the  electronic  catalogs. 

This  step  of  the  process  creates  a  paperless  These  options  are  indicated  in  Figure  4. 

proof  of  delivery  and  is  accomplished  by 

scanning  the  item's  bar  code  label  and  the  The  government  provides  each  ven- 

employees'  (delivery  person  and  dor  with  an  identification  number  that  is  nec- 

customer)  badges  with  hand-held  bar  code  essary  to  access  the  automation  through  a 

scanners.  The  scanned  items  are  security  handler  to  obtain  daily  orders.  Strict 
subsequently  downloaded  into  the  SPEDI  security  is  maintained  to  ensure  that  each 

database.  All  transactions  are  vendor  can  only  access  his  or  her  personal 

automatically  time-stamped  for  reviews,  order  queue.  No  access  is  permitted  to  other 

reports,  or  audits  as  necessary.  vendor  files,  any  other  part  of  the  SPEDI 

database,  or  other  government  automation. 

SPEDI  has  reduced  procurement 

item  lead  time  from  95  days  to  typically  As  previously  mentioned,  financial 

under  3  days  and  has  reduced  payment  to  information  flows  in  and  out  of  SPEDI.  The 

vendors  from  60-90  days  to  10  days  or  input  is  an  ongoing  updating  of  fund  citations 

less.  to  be  used  by  ^e  SOOs  and  as  costing/payment 


FIGURE  4.  SPEDI  Context  Diagram. 
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data.  The  scanning  of  delivered  orders  creates 
the  vendors'  invoices,  which  are  transmitted 
once  a  week  by  electronic  tape  to  the 
comptroller  for  payment.  This  is  one  of  the 
major  benefits  provided  by  the  process  since 
the  vendors'  invoices  also  serve  as  the 
receiving  reports;  therefore,  payment  can  be 
made  immediately. 

In  addition  to  the  items  provided  by 
commercial  contractors,  the  SPEDI  database 
also  contains  all  of  the  items  carried  as  inven¬ 
tory  by  the  Center  Supply  Department,  which 
is  treated  as  another  SPEDI  vendor.  K  an 
item  from  local  stock  is  requested,  the  order 
will  automatically  be  passed  to  the  Supply 
Department  Inventory  Management  submits 


bimonthly  catalog  updates  on  locally  stocked 
items  to  the  SPEDI  database. 

Receiving's  automation  interfaces 
have  been  discussed.  The  SPEDI  database 
administrator  (monitor)  has  unlimited  access 
to  the  automation  and  is  responsible  for 
database  changes,  training  SOOs,  and  cus¬ 
tomer  service.  Other  organizations  can  be 
authorized  limited  access  to  SPEDI  for  on¬ 
line  data  retrieval  and  special  reports. 

Figure  5  shows  the  five  primary 
modules  that  make  up  the  SPEDI  database. 
For  the  most  part,  each  module  deals  with 
one  or  more  of  the  functions  described  in 
Figure  4. 
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FIGURE  5.  SPEDI  Automation  Module  Diagram. 
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ADDITIONAL  BENEFITS 

Electronic  catalogs  are  a  unique  aspect 
of  SPEDL  Each  vendor's  items  are  loaded 
into  the  database  by  Federal  Supply 
Classification,  manufacturer's  part  number, 
local  part  number,  and  key  word(s)  identifier 
which  facilitate  on-line  searches.  Additional 
descriptive  information  may  also  be  viewed 
once  the  item  is  located.  During  the 
solicitation  stage,  floppy  disks  are  provided 
to  offerors  to  be  used  in  submitting  their 
proposals.  Not  only  does  this  expedite  the 
evaluation  process,  but  it  allows  direct 
uploading  of  the  successful  offeror's  items 
into  the  SPEDI  database. 

Software  reporting  capabilities 
include,  but  are  not  lirnited  to: 


Status  of  delivery  orders . On-line 

Utilization  by  fund  citation . On-line 

Utilization  by  customer . On-line 

Utilization  by  shop  code . On-line 

Demand  history . On-line 

Vendor  invoice . Auto-generated 

Delivery  manifest . Auto-generated 

Receiving  report . Auto-generated 

Plant  property  report . Auto-generated 

Vendor  performance . Upon  request 

Vendor  substitutions . Upon  request 

Vendor  price  changes . Upon  request 

Vendor  delinquent  orders . Upon  request 

Orders  for  local 

warehouse  items . Upon  request 

Hazardous  Material  Report . Upon  request 

DD  1057  Report . Monthly 


The  major  benefit  of  the  SPEDI 
database  is  that  it  captures  accurate  real-time 
historical  data.  The  benefits  derived  from  the 
availability  of  such  data  range  from 
determining  precisely  how  much  has  been 
spent  (for  what  items  in  which  organizations) 
to  providing  accurate  data  for  developing  new 
requirements  or  budgeting  purposes. 


SUMMARY 

The  first  SPEDI  prototype  contract 
for  office  supplies  was  awarded  in  October 
1990.  It  took  approximately  1  year  to  bring 
the  contractor  to  a  fully  operational  stage. 


during  which  time  the  hardware,  software, 
and  interfaces  were  demonstrated  to  be 
sound,  as  was  the  bar  code  technology. 

In  October  1991,  a  SPEDI  Operations 
Division  was  formally  established  and  staffed 
with  13  people.  During  its  l-year  existence, 
the  following  has  been  accomplished: 

1.  A  second  SPEDI  contract  for  au¬ 
tomated  data  processing  (ADP)  peripherals 
($2.1  million  for  2  years)  was  awarded. 

2.  Teaming  Partner  Agreements 
(TPAs)  were  established  with  General 
Services  Administration  (GSA)  contractors  to 
supply  ADP  software. 

3.  A  TPA  with  Hewlett  Packard  was 
brought  on  line  under  a  GSA  contract  to 
supply  electronic  test  and  measurement 
equipment 

4.  Three  solicitations  (paint,  safety 
equipment,  and  research  and  development 
chemicals)  are  being  prepared  for  issuance. 

5.  The  option  for  office  supplies  has 
been  exercised. 

6.  Work  has  been  started  on  an  elec¬ 
tronics  requirement. 


NAWCWPNS  purchases  under 
SPEDI  contracts  and  TPAs  for  FY  92  totaled 
over  $3  million. 


CONCLUSIONS 

The  purpose  of  any  investment  is  to 
secure  a  future  gain  that,  even  though  post¬ 
poned,  is  sufficiently  large  to  warrant  the 
earlier  sacrifice.  NAWCWPNS  spent  just 
under  $2  million  to  develop  SPEDI  software; 
this  not  only  included  the  software  develop¬ 
ment,  but  also  the  procurement  support  to  is¬ 
sue  the  first  prototype  contract.  The  yearly 
operating  cost  of  the  SPEDI  division  ( includ¬ 
ing  contractor  automation  support)  is 
$799,000.  Successful  implementation  of  the 
initial  two  contracts  has  proven  that  the  con¬ 
cept  is  stable  and  that  the  software  program 
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and  its  vital  interfaces  perform  as  designed.  It 
was  anticipated  that  by  entering  into  longer 
duration  contracts,  which  included  greatly  ac¬ 
celerated  payment  terms,  quantity  discounts 
of  10%  would  be  attainable.  Presently,  the 
two  contracts  cited  above  are  providing  dis¬ 
counts  that  exceed  40%. 

In  addition  to  the  cost  savings,  signif¬ 
icant  vertical  benefits  result.  These  include: 

1.  Vastly  shortened  procurement  lead 
times  in  the  design  of  new  weapons  systems, 
thereby  lowering  program  schedule  risks. 

2.  Reduced  warehouse  requirements 
and  inventories. 


3.  Reduced  manpower  requirements. 

4.  Increased  technology  information 

flow, 

5.  Better  vendor  accountability  and 
relationships. 

6.  Real-time  data  accumulation  and 
availability. 

In  conclusion,  the  application  of  ex¬ 
changing  data  electronically  used  in  conjunc¬ 
tion  with  today's  bar  code  technology  offers 
endless  opportunities  to  improve  business 
processes  by  better  utilizing  resources,  accel¬ 
erating  transactions,  and  providing  real-time 
data  upon  which  to  base  critical  decisions. 
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IS  DARPA  PREJUDICIALLY  FAVORING  THINKING  MACHINES? 

Jon  R.  Troyer,  Grad.  Asst.,  University  of  Dallas,  GSM 


ABSTRACT 

Such  a  charge  was  leveled  against 
DARPA  by  supercomputer  manufacturers,  so 
"sales  don't  come  down  to  technology  or 
software,  [but]  to  someone  playing  God  and 
saying  we'll  make  it  easier  to  work  with  one 
vendor."'  Were  it  not  for  the  timeliness  of 
several  issues  raised  by  the  question  "Is  Dar- 
pa  Prejudicially  Favoring  Thinking  Machi¬ 
nes?"  echoes  of  laughter  would  close 
discussion.  The  U.S.  needs  policies  to  spur 
innovation  that  are  lacking  in  the  current 
competition  mandates.' 

Can  savings  that  are  achieved 
only  from  innovation  be  exceeded  by  savings 
from  the  competitive  selection  of  items  which 
otherwise  would  not  be  manufactured?  As¬ 
sumptions  of  saving  on  the  basis  of  competi¬ 
tion  are  wrong  The  appearance  of 
favoritism  is  due  to  misunderstanding  the  na¬ 
ture  of  markets,  which  are  arbitrary  group¬ 
ings,  and  failure  to  compare  savings  from 
competition  to  savings  from  innovation 

Congress  "tends  to  think  of  itself 
as  a  body  that  not  merely  provides  some  ser¬ 
vices  for  an  independently  functioning  order 
but  'runs  the  country'  as  one  runs  a  factory  or 
any  other  organization.  Since  it  possesses 
authority  to  arrange  everything,  it  cannot  re¬ 
fuse  responsibility  for  anything.""  Congres¬ 
sional  distrust  of  sole-source  profiteering 
should  be  more  than  counterbalanced  by 


fears  of  subsidized  stagnation,  as  savings 
from  innovation  are  greater  than  savings 
from  competition.  Ironically,  boosting  com¬ 
petitiveness  is  done  simply  by  repeal  of  the 
Competition  in  Contracting  Act,  because 
breakthrough  thinking  spurs  growth. - 

INTRODUCTION 

This  paper  supports  the  enact¬ 
ment  of  an  "Innovation  In  Contracting  Act." 
The  Innovation  in  Contracting  Act  key  fea¬ 
tures  are  three  forms  of  procurement,  1)  in¬ 
novative  proposals,  2)  sealed-bidding,  and  3) 
catalogue  purchases.  The  sealed-bid  awards 
are  priced  by  competition.  The  innovative 
proposals  are  priced  by  virtual  competition 
because  after  five  years  the  agency  would  be 
required  to  compete  innovative  items  in  a 
sealed-bid,  and  so  fully  market  price  the 
items  A  Contract  Renegotiation  Board 
would  be  required  to  limit  unit  price  during 
the  five  year  period  to  less  than  three  times 
the  awarded  open  market  price.  This  pre¬ 
cludes  any  and  all  requirements  for  cost  or 
pricing  data.  Multi-step  sealed  bidding  will 
replace  all  competitive  proposals,  setting  a 
market  price  by  auction  competing  suppliers 
without  cost  analysis.  Noncompetitive  pro¬ 
posals,  unsolicited  proposals,  sole-source 
awards,  or  any  award  where  there  is  one  in¬ 
novative  supplier  are  called  innovative  pro¬ 
posals. 


'  "Pentagon  Unit  Steers  Supercomputer  Deals  to  Certain  Companies  -  Rival  Firms  Complain  Darpa  is  Un¬ 

fair  in  the  Way  It  Stresses  Parallel  Processing  -  Agency  that  Hates  Red  Tape"  (1991)  77;c  Hi///  Street  Journa/.  6 
August,  pp.  A1,A6. 

”  Hayck.  F  A.  (197.1) /.me,  Legis/alian  and  I.it/ertw  U  of  Chicago  Press,  p.  14.1 
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The  purpose  is  to  stimulate  in¬ 
novation  beyond  product  differentiation,  and 
reinforce  careful  thinking  about  markets.  In 
short,  the  whole  procurement  process  gains 
by  opening  the  world  to  the  radical  departure 
of  Darpa  and  Thinking  Machines. 

On  its  face,  favored  treatment  by 
the  government  in  any  procurement  could 
limit  competition  through  the  application 
bribes,  and  other  conspiracies  to  defraud  by 
using  public  office  for  private  gain.  In  fact 
the  adversarial  environment  of  public  pro¬ 
curement,  makes  this  a  remote,  tenuous, 
even  humorous  charge 

Such  a  charge  was  leveled  by  su¬ 
percomputer  manufactures  including  Convex 
of  Richardson,  Texas,  Cray  Research,  and 
NCube.  They  are  lobbying  for  competitive 
bidding  on  supercomputer  purchases,  and  ac¬ 
celerated  purchase  contracts.  What  emerges 
are  issues  bound  to  national  security,  the 
economic  welfare  of  the  U.S  .,  and  faulty  fed¬ 
eral  procurement  law.  Defense  R&D,  the 
U  S.  edge  in  high  technology,  is  increasingly 
important  so  that  d'^astic  measures  are  not  re¬ 
quired  if  it  is  necessary  to  reverse  the  current 
drawdown.  Competition  Advocacy  is 
doomed  because  it  seeks  to  create  sources, 
not  to  spur  innovation,  but  only  for  the  sake 
of  obtaining  two  bids,  which  is  more  than 
one.  Those  who  enter  the  market  after  R&D 
has  been  completed  do  not  require  incentive 
prodding,  but  restriction  against  using  pro¬ 
prietary  data  and  so  dousing  innovation.’ 

Now  let's  backup  to  examine 
Thinking  Machines  and  Darpa,  The  Competi¬ 
tion  in  Contracting  Act  of  1984,  Competition 
and  pre-competition,  and  then  an  analysis  In¬ 
novation  in  Contracting  applied  to  case  law. 


1.  DARPA  AND  THINKING  MACHINES 

The  prototype  Connection  Ma¬ 
chine  was  funded  under  contract 
#N00039-84-C-0638.  In  the  acknowledge¬ 
ments  section  of  his  dissertation  Daniel  W. 
Hillis  gives  tribute  to  "Craig  Fields,  who 
knew  it  was  the  right  thing."'  Recently  Hillis 
announced  a  new  machine  that  departs  from 
the  original  conception,  but  which  appears  to 
be  winner  of  the  LINPACK  WARS  or  race 
to  develop  a  machine  that  exceeds  a 
TeraFLOP.^  Previously  Thinking  Machines 
announced  surpassing  nine  GigaFLOPS.' 
Darpa  made  a  $12  million  award  in  1 989,  at 
the  same  time  awarding  Intel  $7.7  million/’ 

The  Strategic  Computing  Pro¬ 
gram  is  program  is  duel  purpose  in  the  sense 
that  the  applications  are  both  military  and 
commercial.’  NCube  has  complained  about 
not  getting  a  subsidy  for  a  commercial 
customer  *  The  CM2  at  Mobile  R&D  in 
Farmers  Branch,  and  the  Convex  machine 
beside  it  were  purchased  without  government 
support.  Aeronautical  design  programs  or 
battlefield  programs  have  features  common 
to  seismic  exploration  programs.  This  is 
hardly  venal.  Darpa  spends  in  excess  to 
$100  million  of  its  $1.5  Billion  in  the  Strate¬ 
gic  Computing  Program. 

Darpa  was  created  by  PL  85-325 
responding  to  Sputnik.’  Darpa  programs  in¬ 
clude  light  satellites,  graphics  workstations, 
artificial  intelligence,  composite  materials, 
digital  gallium  arsenide,  high-density  dis¬ 
plays,  global  positioning  system  enhance¬ 
ments  and  anti-helicopter  mines  infrared 
beacons  for  Desert  Storm,  ultra  wide-band 
(B-2  killing)  radar.  Darpa  also  developed  the 
automatic  teller  machine  and  the  PC  mouse 
The  X-3I  National  Space  Plane  is  pioneering 


'  Hillis.  W.  Daniel.  (1989)  The  Connection  Machine,  The  MIT  Press.  1989.  Same  as  Ihcsis  (Ph  D  )-MIT. 

1985. 
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new  research  frontiers.  The  Strategic  Com¬ 
puting  Program  aims  to  solve  the  Grand 
Challenges  including,  "global  weather  predic¬ 
tion,  determining  atomic  structures,  under 
standing  turbulence,  mapping  the  human  ge¬ 
nome."' 


Congress  authorized  "flexible 
agreements,"  and  Craig  Fields  gave  $4  mil¬ 
lion  to  Gazelle  Microcircuits,  which  John  Su- 
nunu  new  was  the  wrong  thing.  But  the 
White  House  is  softening  on  so  called  Indus¬ 
trial  Policy,  and  the  program  appears  to  be 
continuing. 

The  Darpa  Initiative  in  Concur¬ 
rent  Engineering  is  an  effort  shorten  time  to 
production.  The  DOD  has  mandated  the 
Computer-Aided  Acquisitions  and  Logistics 
Support  program  (CALS)  which  includes 
data  interchange  and  design  tool  specifica¬ 
tions  for  improving  acquisition  and  program 
support."  Concurrent  Engineer*';  poses  a 
new  opportunity  to  redefine  cost  or  pricing 
data,  especially  for  paper-driven  bureaucra¬ 
cies.  Ultimately,  that  is  the  last  thing  needed 
by  an  innovation  fostering  policy,  and  signals 
the  end  to  the  usefulness  of  acquisition  cost 
or  pricing  data.  Inclusion  of  work  measure¬ 
ment  system  data  mandates  disclosure  of 
measurements  made  for  variance  analysis, 
and  how  process  planning,  cost  estimating, 
and  productivity  improvement  is  done  as  cost 
or  pricing  data.  The  courts  may  have  the  op¬ 
portunity  to  read-in  a  legal  definition  of 
TQM  when  DCAA  conspires  with  DCMC 
Quality  Division  for  enforcement.  What  a 
great  way  to  send  Deming  to  his  grave. 
Roadblocks  are  eliminated. 


2.  COMPETITION  IN  CONTRACTING 
ACT  FLAWED 

CICA  is  flawed.  Convex,  Cray, 
and  N-Cube  maintain  they  are  in  the  same 
market  characterized  by  high-speed  comput¬ 
ing  power.  This  is  an  arbitrary  assumption, 
i.e.  that  competitors  in  LINPACK  WARS 
comprise  a  market  grouping.  Supercomput¬ 
ers  are  not  standardized  or  easily  substituted. 
The  differences  between  products  amount  to 
more  than  the  differentiation  between  Chevy 
and  the  Cadillac.  The  grouping  is  convenient 
only  from  the  stand-point  decreasing  con¬ 
sumer  confusion,  though  the  products  "Con¬ 
sumer  Reports"  would  make  comparison 
much  less  sure.  For  instance  the  CM-2  was 
optimized  for  fine-grain  parallelism,  short 
words,  many  processors,  and  less  memory 
per  processor,  while  the  Intel  Machine  opti¬ 
mized  for  fewer  processors,  wide  words,  and 
more  memory  per  processor."'^"  These 
vast  differences  mark  more  than  differenti¬ 
ation,  and  question  seriously  whether  they 
are  in  the  same  market. 

A  sealed-bid  for  supercomputers 
unravels  another  problem  with  CICA,  and 
Comptroller  General  (Comp  Gen)  cases  be¬ 
fore  it,  specifications  unduly  restrictive  of 
competition,  but  is  resolved  by  understanding 
that  markets  are  arbitrarily  grouped. 

A  sealed-bid  for  the  "Connection 
Machine  or  equal"  would  surely  be  found  re¬ 
strictive  of  competition,  because  everyone 
knows  a  supercomputer  is  one  thing,  a  LIN¬ 
PACK  measurement. 

If  the  needs  of  an  agency  are 
clearly  defined  discriminating  selections  are  a 


'  Mygatt,  Matt.  (1991)  "Supercomputer  experts  gathertodiscuss  their  trade,"  17Novembcr.  (Wil¬ 

liam  Comp,  manager  of  mathematics  and  computational  «;ience,  Los  Alamos,  further  states.  "We  would  like  to 
help  the  U.S.  industry  become  more  competitive  in  the  economy  by  helping  them  understand  what  we  have  lear¬ 
ned") 
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good  practice.  All  that  essential  features  and 
minimum  needs  buy  are  products  void  of 
specialized  factors  of  production,  the  lowest 
common  denominator,  the  most  stagnant 
technology,  the  undifferentiated  in  a  Freudian 
sense. 

fails  to  strongly  emphasize 
that  the  foundation  of  procurement  is  well 
written  specifications.  Cibinic  considers 
specifications  the  "cornerstone  of  competi¬ 
tive  procurement."'  Well  written,  specifically 
defined  specifications  lead  to  fair  evaluations 
of  subjective  factors,  because  the  factors  are 
well  understood  by  all  competitive  bidders, 
called  the  market.  Well  written  specification 
lead  to  fair  evaluation  of  innovative  subjec¬ 
tive  factors  which  are  well  understood  by  one 
supplier,  called  the  pre-market  or  pre¬ 
competition. 

If  the  needs  of  an  agency  are 
clearly  defined  it  is  an  unfortunate  misunder¬ 
standing  of  markets  to  find  that  the  agency's 
minimum  needs  have  been  overly  restrictive 
of  competition.  That  is  based  on  an  arbitrary 
market  grouping,  and  death  to  innovation. 
Poorly  written  solicitations  leave  no  doubt  as 
to  market  ignorance.  Well  written  specifica¬ 
tions  lead  to  fair  evaluations  of  subjective 
factors,  because  the  factors  are  well  under¬ 
stood  by  all  bidders. 

Why  is  Competition  important? 
Former  Vice-President  Quayle  was  head  of 
the  Competitiveness  Council,  so  maybe  its 
not.'®  Actually  Quayle  as  a  Congressman 
signed  a  letter  to  the  DOD  which  softened 
DOD  implementation  of  CICA  regarding 
R&D  money.  ^  Competition  is  important  to 
achieve  the  best  possible  utility  for  the  buyer 
and  the  seller.  Competition  is  an  auction 
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where  buyers  and  sellers  reverse  the  tradi¬ 
tional  role  to  gain  the  best  bargain  possible 
which  the  lowest  transaction  costs.  The  con¬ 
sumer  and  supplier  surplus,  the  excess  gain 
of  utility  over  the  final  bargain  is  wasted  by 
competitive  proposals,  obliterated  by  huge 
transaction  costs.  Although  competitive  pro¬ 
posal  were  brought  to  the  same  legal  stand¬ 
ing  as  seal-bids  the  economic  differences  are 
not  reduced.  But  even  Milgrom  plays  the 
tune  of  "possibilities  for  influence,  favoritism, 
and  bribes"  as  if  Fields  must  be  swimming  in 
filthy  kick-back  lucre" 

More  than  one  proposal  defining 
approaches  to  the  identical  requirement  is 
competitive  In  the  five  years  after  CICA 
competitive  procurements  amounted  to  less 
than  forty  percent  of  all  DOD  contract 
dollars.”  This  indicates  poorly  defined  re¬ 
quirements,  and  huge  economic  losses  on 
sole-source  procurement  which  rely  on  virtu¬ 
ally  no  market  prices.  Generally,  government 
cost-price  analysts  are  loathe  to  use  pricing 
analysis  (Because  they  receive  taxpayer 
back-breaking  gratuities'^) 

The  CICA  provision  that  all  un¬ 
solicited  proposals  be  market  tested  has  set  a 
policy  ruinous  to  innovation,  and  for  the 
flimsy  reason  that  agencies  may  be  flooded 
with  proposals  protected  from  competition  '* 

The  result  is  that  a  contracting 
officer  will  put  proprietary,  or  slightly  dis¬ 
guised  proprietary  ideas  and  innovations  out 
on  the  street.  Nothing  could  scuttle  innova¬ 
tive  ideas  faster 

After  understanding  the  essential¬ 
ly  arbitrary  nature  of  markets,  strictly  speak- 
i.ig  every  manufacturer  could  make  the 


'  ( 1982)  Senate  Report  no.  98-50.  USCCANS9-2.  p.  2187, 

"  Milgrom.  Paul.  (1989)  Auctions  and  Bidding:  A  Primer,  Journal  of  Economic  PerspecUves.  Vol.  .^.  No  .1, 

Summer,  pp.  3-22.  (Analysis  of  "Winners  Curse."  when  you  win.  you  lose.) 
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previously  feared  "unique  and  innovative" 
claim  for  protection  from  competition.  But 
hypothesize  that  the  Contracting  Officer  is 
not  required  to  market  test  unsolicited  pro¬ 
posals.  A  and  B  both  make  proprietary 
claims  for  items  that  are  easily  substituted, 
and  the  Contracting  Officer  cannot  put  these 
on  the  street  What  does  he  do"^  He  com¬ 
petes  A  against  B  behind  closed  doors,  and 
entertains  bids/proposals  by  easily  substituted 
X,  Y,  Z.  He  picks  the  best  one  of  the  five  re¬ 
sulting  in  no  impact  on  cost  Therefore,  this 
provision  is  unreasonable.  And  by  blocking 
innovation  the  provision  for  market  testing  of 
unsolicited  proposals  is  disastrous. 

Stimulating  innovation  is  a  legiti¬ 
mate  policy.  Innovation  is  savings.  Dr.  Da¬ 
vid  Gordon,  U.  of  Dallas,  "You  are  worse  off 
to  spend  money  on  a  system  if  the  systems 
really  should  not  be  produced  in  the  first 
place."  To  many  companies  are  kept  busy 
protecting  there  markets,  or  asking  regu¬ 
lators  to  keep  them  profitable,  and  so  innova¬ 
tion  is  kept  a  low  priority  The  competitive 
advantage  derived  from  innovation  springs 
from  up-grading  a  process  or  procedure  so 
that  it  realizes  efficiency  which  is  difficult  to 
duplicate.''’-"  This  should  be  heartily  encour¬ 
aged  in  all  fields  of  endeavor,  among  numer¬ 
ous  companies.  A  truly  better  widget 
develops  the  firm's  and  the  countries  re¬ 
source  base  Better  there  be  more  innovative 
contracts  than  competitive  ones,  and  for 
more  dollars,  to  drive  down  costs.  Darpa 
successfully  does  this  in  many  advanced  pro¬ 
grams,  but  that  is  only  the  tip  of  the  ice-berg. 
Describing  the  latest  function,  Lee  Buchanan 
director  of  the  Defense  Sciences  Office  at 
Darpa,  is  sufficiently  fearful  of  consequences, 
"Darpa  is  not  consciously  acting  as  a  venture 
capitalist,  although  it  would  appear  that  we 


are  to  the  company  involved  because  we  will 
p.articipate  in  their  profits.  However,  Darpa 
will  not  control  stock,  will  not  be  involved  in 
the  management  of  the  business  and  will  not 
be  involved  in  setting  industrial  policy  "'  He 
is  following  the  same  Congressionally  funded 
program  which  lead  to  Field's  departure. 

Rather  than  creating  separate 
agencies  and  functions  to  support  innovation, 
mandate  it  across  the  board.  These  ideas  are 
elegantly  expressed  by  Porter  who  calls  for 
policy  to  transcend  to  a  "higher  plane,""  and 
signals  the  resolution  of  the  Industrial  Policy 
Debate,  which  we  redefine  in  terms  of  com¬ 
petition  and  pre-competition. 

3.  PRE-COMPETITION 

The  free-marketers  and  the  inter¬ 
ventionists  are  equally  wrong,  but  what  are 
they  saying.  Free-marketers  are  against  gov¬ 
ernment  planning,  rent  control,-'  wetlands 
management,--  protectionism,  and  cartels  that 
protect  the  inefficiency  of  big  companies.  In¬ 
terventionists  like  to  say  MITI  Japanese  gov¬ 
ernment  subsidies  account  for  Japanese 
success,  and  that  cooperative  programs,  tax 
incentives,  and  loan  guarantees  will  solve  the 
riddle  of  competitiveness  (outselling  the 
Japanese. p 

Craig  Fields  now  works  at  Micro¬ 
electronics  and  Computer  Technology  Corp. 
in  Austin,  where  he  hopes  to  prove  that  pub¬ 
lic  money  ($100  million)  for  advanced  re¬ 
search  can  improve  the  economy. -•*  His 
"retirement"  from  government  service  ‘'April 
1990)  was  seen  as  a  victory  for  the 
free-marketers.-''  But  in  March  1990  Bush 
had  promised  support  for  both  commercial 
and  defense  research.  HDTV,  with  wide 


'  Henderson,  Breek  \V  (1991)  "Darpa  Invests  $4  million  As  Venture  Capital  in  High  Technolog\’  Compan¬ 
ies." /I  Week  and  Space  Technology.  ;t0  April,  p.25. 

"  "Industrial  Policy"  (1992)  Bu.siness  Week.  6  April,  p.73. 
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commercial  application,  is  currently  funded 
by  Darpa  for  $70  million.-® 

Commercial  spin-offs  from  mili¬ 
tary  research  are  under-whelming.-’  The  Na¬ 
tional  Institute  of  Standards  and  Technology 
(NIST)  was  established  from  the  old  Nation¬ 
al  Bureau  of  Standards  in  1988,  and  has 
some  Darpa-like  coinmertial  functions,  but  is 
under-funded.-®  The  National  Science 
Foundation  (NSF)  supports  basic  research 
but  there  has  been  a  call  for  a  separate 
agency  to  support  productivity  research 
"virtually  ignored  and  left  to  the  business 
schools."'  It  is  under-educated. 

Most  military  R&D  is  allocated  to 
specific  weapons  programs,  and  as  those 
programs  decline,  the  R&D  declines  faster. 
If  military  and  commercial  advanced 
technologies  are  the  same,  less  military  re¬ 
search  also  means  less  commercial  research, 
and  the  point  is  that  the  decline  in  military  re¬ 
search  should  be  mirrored  by  increased  com¬ 
mercial  spending  As  defense  declines,  NIST 
and  NSF  should  increase.-* 

It  is  possible  that  government 
spending  on  basic  research  diverts  a  given 
companies  attention  from  crucial  proprietary 
research,  but  the  trade-off  is  an  increase  in 
the  range  of  improvements  and  special 
knowledge  of  the  factors  of  production,  a 
subsidy  for  the  shot-gun  approach  to  innova¬ 
tion  called  pre-competition  at  the  national 
level. 

The  government  has  the  responsi¬ 
bility  to  provide  a  technically  current  econo¬ 
my,  so  that  its  autonomous  or  self-directed 
citizens  can  successfully  execute  their  plans. 


Laizez-faire  needn't  be  a  tyrant,  nor  Indus¬ 
trial  Policy  the  end  of  individualism  If  you 
are  autonomous,  you  innovate,  and  converse¬ 
ly,  depending  in  great  part  on  the  contracting 
capabilities.’’'^®-^' 

4.  INNOVATION  IN  CONTRACTING  V. 
CASE  LAW 

"We  now  have  a  new  statutory  approach  as 
well  as  statutory  codification  of  competitive 
concepts  developed  over  the  years  by  the 
Comptroller  General,"  Doke." 

Starting  with  this  well  informed 
judgement  (w/  generous  a  fact  for  students 
of  mumbo-jumbo  cost  or  pricing  data)  of 
CICA's  arrival,  let's  see  if  the  proposed  In¬ 
novation  in  Contracting  Act  would  sit  well 
with  case  law  (authority).  All  the  cases  are 
from  Nash  and  Cibinic"'  To  reiterate  the  key 
features  of  IICA  it  includes  three  forms  of 
procurement,  1)  innovative  proposals,  2) 
sealed  bid,  and  3)  catalogue  purchases.  The 
sealed-bid  awards  are  priced  by  competing 
bidders.  The  innovative  proposals  are  priced 
by  virtual  competition  because  after  five 
years  the  agency  would  be  required  to  com¬ 
pete  innovative  items  in  a  sealed-bid,  and  so 
market  price  the  items  A  Contract  Rene¬ 
gotiation  Board  would  be  required  to  limit 
unit  price  during  the  five  year  period  (gener¬ 
ous  because  product  cycles  are  now  less  than 
three  years)  to  less  than  three  times  the  open- 
market  price.  This  precludes  any  and  all  re¬ 
quirements  for  cost  and  pricing  data.  Cata¬ 
logue  purchase  rules  follow  commercial 
practices  iUCC). 


'  Brinkman,  W.F.  (1990)  "A  National  Engineering  and  Technology  Agency."  Science,  February  ,  p.  901. 

(From  the  scrap  heap  of  science.) 

"  Doke,  Marshall  J.,  Jr.  (1985)  "Recent  Legislative  Developments  in  Government  Contracts,"  M)  January 
(Subjective  evaluation  factors  may  allow  sclectivity/caprice  by  agencies.) 

Nash  and  Cibinic  (1976)  Government  Contract  Law.  George  Washington  University,  pp.  19.^-381 
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Contractor  burdens  continue  to 
include  responsiveness  and  responsibility. 
Government  burdens  of  full  and  fair  competi¬ 
tion  are  altered  to  induce  innovation,  to 
stimulate  growth.  Clarity  of  specs  is  empha¬ 
sized.  Other  than  the  following  judicial  au¬ 
thority,  Comp  Gen  included,  are  at  times 
made  defunct  by  subsequent  statutes. 

Contractor  Responsiveness 

The  Comptroller  General  rule  is 
that  as  long  as  substance  (price,  quantity, 
quality)  is  unchanged,  deviations  are  allowed. 
Statutes  disallow  acceptance  of  non- 
responsive  bids  Because  it  is  likely  that 
more  sealed-bids  will  be  done,  and  respon¬ 
siveness  is  required  in  sealed-bids,  respon¬ 
siveness  will  gain  a  higher  profile. 
Responsiveness  in  the  case  of  innovative  pro¬ 
posals,  would,  like  the  present  non¬ 
competitive  awards,  not  be  formally  required. 
Prestex  v.  USA  G 963). 

Determinations  of  responsiveness 
must  be  in  the  government's  best  interest  and 
without  prejudicial  favoritism.  In  responsive¬ 
ness  disputes,  sadly,  charges  of  favoritism 
will  remain  a  factor  under  IICA.  Comp  Gen 
B- 148624  (1962). 

Proof  of  reliability  would  contin¬ 
ue  to  be  included  as  a  responsiveness  deter¬ 
mination,  separate  from  predicting 
responsibility.  Though  responsiveness  is  not 
an  issue  of  innovative  proposals,  like  nego¬ 
tiated  proposals,  proof  of  a  proposed  prod¬ 
ucts  capability  should  be  demonstrated. 
Comp  Gen  B-1 75492(1)  (1972). 

The  necessity  for  substantial  work 
to  upgrade  a  proposal  to  level  of  responsive 
which  can  be  a  finding  of  non-responsiveness 
could  be  the  source  of  an  end-run  should  the 
contractor  be  able  to  show  that  the  contract 


contemplated  was  obsolete  and  though  not 
responsive  to  the  contemplated  action  could 
perform  the  agencies  needs  under  an  innova¬ 
tive  proposal.  Possible  if  basic  specs  are  ex¬ 
ceeded  or  somehow  themselves  obsolete. 
Innovative  proposals  could  easily  begin  with 
the  Non-Inclusive  clause  which  states  noth¬ 
ing  shall  prohibit  the  agency  finding  another 
method  to  achieve  the  contemplated  goals  of 
the  solicitation.  PRC  Computer  (1975). 

The  computation  of  weighting 
factors  which  are  misleading,  or  that  reflect 
that  more  than  price  is  at  issue,  or  that 
change  the  specs  to  upgrade  for  responsive¬ 
ness  without  telling  all  bidders,  indicate  an 
arbitrary  decision.  Setting  a  market  price  is 
the  goal  in  sealed-bidding,  but  there  are  many 
pitfalls.  Arbitrary  abuse  of  authority  is  dis¬ 
putable  in  the  contractor's  world  and  would 
continue  to  be.  Bell  Aerospace  (1975) 

Contractor  Responsibility 

The  contracting  officer  has  the 
duty  to  resolve  inconsistencies  and  uncertain¬ 
ties  regarding  responsibility.  Successful  pro¬ 
test  of  Contracting  Officer  n  n-responsibility 
finding.  For  sealed-bid  this  is  the  same,  and 
it  would  be  a  requirement  for  innovative  pro¬ 
posals.  Comp  Gen  B-172061  (1973). 

There  must  be  some  evidence  the 
contractor  is  responsible.  It  is  possible  to 
protest  Contracting  Officer  affirmative  find¬ 
ing  of  responsibility,  though  less  likely  of 
success  than  the  protest  of  the  negative  find¬ 
ing  above.  Responsibility  is  a  factor  under 
any  scheme,  no  change  for  IIC  Data  Test 
Corp,  (1975). 

Government  Specification 

A  mixture  of  performance  specs 
and  detailed  technical  specs  caused  confusion 
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to  competing  parties.  Specs  should  be  well- 
written.  Without  careful  purchase  planning 
sealed-bids  and  innovative  proposals  are  use¬ 
less.  Contractors  should  always  be  given  the 
opportunity  to  clarify  specs.  Sealed-bids 
should  be  conducted  so  that  there  is  ample 
opportunity  to  clarify  specs.  Comp  Gen 
B-175585  (1971). 

Actual  requirements  must  be 
present  in  an  auction,  or  innovative  proposal. 
If  there  is  indecision  about  what  criteria  are 
to  be  applied  there  can  be  no  mutuality  of 
agreement  let  alone  competition  or  innova¬ 
tion.  Refining  Associates  v.  USA  (1953). 

There  must  be  certainty  as  to 
what  price  is  actually  being  bid  in  a  sealed- 
bid.  It  should  be  possible  to  include  all  con¬ 
tract  types  under  sealed-bids,  if  it  is  com¬ 
monly  understood  what  is  being  bid  on  and 
commonly  understood  what  the  price  bid  is. 
Comp  Gen  B-178192  (1973). 

Two-step  sealed  bidding  should 
enjoy  a  revival,  especially  where  a  highly  spe¬ 
cialized  process  has  been  achieved  by  a  larger 
group  of  competitors  (gains  from  innovation 
are  not  generally  an  infinity,  e  g.  Mrs. 
Baird's  Bread,  or  the  CM-2  five  years  hence.) 
It  would  be  unfair  if  bidders  were  bidding  on 
different  solicitations.  Comp  Gen  B-178192 
(1973). 

The  specs  cannot  be  altered,  un¬ 
less  they  are  waived  for  all  bidders,  which 
means  the  prospective  agreement  is  changed. 
Again  all  bidders  must  be  bidding  on  the 
same  thing.  Alteration  of  an  innovative  pro¬ 
posal  must  also  be  specific  about  changes  to 
maintain  clear  understanding  of  the  program, 
Corbetta  Const.  (1976). 


Competition  Rulings 

The  requirement  for  meeting 
minimum  needs  so  as  to  maximize  competi¬ 
tion,  should  be  recast  so  as  to  maximize  in¬ 
novation  while  fulfilling  the  agency's  needs. 
Minimum  needs  are  not  discriminating  needs, 
but  only  needs  essential  to  agency  require¬ 
ments.  In  contrast  IIC  would  make  meeting 
discriminating  needs  a  top  agency  priority. 
This  purchase  would  be  a  simple  catalogue 
purchase,  pre-priced.  Comp  Gen  B- 164993 
(1968). 

Performance  requirements  should 
be  accessible  and  inclusive,  without  "brand- 
name  or  equal"  which  is  restrictive  of  com¬ 
petition.  The  finding  is  based  on  specs 
which  were  adequately  defined  "Brand- 
name  or  equal"  is  only  appropriate  where 
there  are  not  good  specs  available.  It  should 
never  be  appropriate  Comp  Gen  B- 166849 
(1969). 

Competition  is  not  unreasonably  re¬ 
stricted  because  some  cannot  meet  require¬ 
ments.  Responsiveness  and  responsibility 
determinations  must  be  made  in  good  faith. 
The  sections  above  described  the  responsive¬ 
ness  and  responsibility  burdens  on  contrac¬ 
tors.  Comp  Gen  B- 175254(1)  (1973). 

Legitimate  needs  must  not  be 
compromised  to  obtain  competition,  but  if  no 
development  is  required  and  items  are  off- 
the-shelf,  solicitations  must  be  made  avail¬ 
able,  Under  IIC  the  requirement  for  Fluke- 
meters  would  have  automatically  been  bid 
competitively  in  five  years,  or  simply  ordered 
from  the  Fluke  catalogue  Non-Linear  Sys¬ 
tems  (1975). 
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5.  ASK  CRAIG  FIELDS 

If  challenges  to  arbitrary  abuse  of 
authority  were  also  disputable  within  govern¬ 
ment  ranks,  if  there  existed  an  equitable  es- 
topple  to  uphold  wavering  reliance,  and 
reverse  official  mismanagement,  government 
worker-bees  would  halt  many  inequities  be¬ 
fore  they  had  to  be  moved  to  a  another  juris¬ 
diction.  According  to  Dan  Riley,  Adjunct 
Professor  at  the  U.  of  Dallas,  and  Partner  in 
the  law  firm  Doke  and  Riley,  "Estopping  the 
government  without  a  contract  is  ...  diffi¬ 
cult."  That  is  the  century's  understatement. 
Ask  the  conscientious  Craig  Fields  or  his 
supporters  about  sovereignty  not  held  ac¬ 
countable  by  the  arm's  length  contract. 

From  the  stand  point  of  current 
law,  ruinous  to  innovation.  Fields  did  not 
simply  extend  prejudicially  favoring  terms. 
Fields  held  Hillis  above  the  law,  and  Fields 
was  later  punished  foi  similar  action.  But 
rather  than  shining  a  light  on  a  corrupt  pro¬ 
curement,  the  story  illuminates  the  gross  in¬ 
adequacy  of  current  law.  It  is  the  mission  of 
Darpa  to  stimulate  advanced  research,  but 
because  innovation  should  be  the  preoccupa¬ 
tion  of  every  company,  not  just  Hillis',  and  at 
the  core  of  every  contract,  these  lessons  must 
gain  wide  subscription,  mandated. 

SUMMARY 

Because  of  the  value  of  expecta¬ 
tions,  the  spirit  and  bone  of  a  new  century 
begins  a  decade  prior.  The  world  wide  re¬ 
cession  with  global  consolidations  tells  us 
our  first  breaths  of  the  2 1  st  Century  are  best 
symbolized  by  walking  pneumonia. 

Innovation  in  Contracting  sends 
the  message  that  saving  by  innovation  gets 
priority.  It  also  pays  a  market  price.  The  use 
of  cost  or  pricing  data  is  inferior,  and  de¬ 


stroys  autonomy,  by  saying  that  though  we 
trust  you  to  build  point,  click,  and  kill  in  N+1 
dimensions,  we  just  don't  trust  you  with 
$2.50. 
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COST  AND  SCHEDULE  EXPERIENCE  FOR  GROUND  COMBAT  AND  SHIP 

ACQUISITION  PROGRAMS 

Karen  W.  Tyson,  Neang  I.  Om,  D.  Calvin  Gogerty,  and  J.  Richard  Nelson, 

Institute  for  D^ense  Analyses 


This  paper  highlights  the  cost  and  schedule 
experience  of  programs  to  acquire  ground  combat 
systems  and  ships  since  the  1960s.  This  is  important, 
because  it  gives  us  a  picture  of  the  effectiveness  of 
acquisition  policy  for  these  groups  of  equipment  over 
time. 

Ground  combat  programs  include  military 
vehicles  and  a  set  of  distinctive  Army  programs.  The 
Navy  ships  and  related  acquisition  programs 
comprise  three  combat  data  systems,  ten  classes  of 
surface  vessels,  and  two  classes  of  attack  submarines. 

We  will  see  that  the  two  groups  of  systems 
had  dramatically  different  cost  and  schedule 
outcomes.  In  order  to  explain  the  differences,  we 
examined  whether  management  initiatives  resulted 
in  more  favorable  outcomes  for  each  group  of 
systems.  We  also  considered  distinctive  features  of 
the  two  program  groups  that  might  account  for  the 
differences. 

I.  DATA 

We  obtained  information  on  development 
and  production  costs  and  schedules  and  IOC  dates 
from  the  Selected  Acquisition  Reports  (SARs)  for 
each  program. 

For  some  ground  combat  systems  that  have 
multiple  major  modiiications  (e.g.,  Bradley  FVS)  or 
multiple  subsystems  [e.g.,  rocket  and  launcher  for 
the  Multiple-Launch  Rocket  System  (MLRS)],  the 
SAR  does  not  have  development  cost  nor  production 
cost  reported  by  modification  version  or  by  rocket 
and  launcher.  In  these  cases,  we  obtained  the 
breakdown  of  production  costs  by  version  and 
subsystem  from  the  Army  Cost  and  Economic 
Analysis  Center  (CEAC).  For  siuface-Iaimched 
munitions  systems  (e.g.,  MLRS),  price  improvement 
curves  for  the  cunent  estimate  of  production  cost  at 
Development  Estimate  Quantity  were  developed  by 
launcher,  rocket,  and  practice  rocket  and  added 


together  to  obtain  the  total  production  cost.  In  the 
case  of  the  ships,  we  supplemented  the  SAR  data 
with  information  supplied  by  the  Navy  Center  for 
Cost  Analysis  and  with  data  from  technical 
references  such  as  Jane's  Fighting  Ships. 

Development  or  current  estimates  of 
production  costs  for  the  air  defense  and  submarine 
combat  systems  could  not  be  determined  from  the 
SARs.  Any  production  cost  data  shown  in  the  SARs 
for  those  programs  are  for  support  equipment  The 
actual  production  costs  for  those  programs  are 
included  in  the  production  costs  for  the  ships  in 
which  they  are  installed,  and  those  costs  cannot  be 
broken  out  separately  from  the  SARs  for  the  ships. 

Cost  growth  is  defined  as  the  percentage  by 
which  actual  cost  exceeds  estimated  cost.  (Negative 
growth  is,  of  course,  possible,  if  the  actual  turns  out 
to  be  less  than  the  estimate.)  In  development,  actual 
cost  is  measured  from  the  beginning  of  development 
to  the  end  of  development  of  the  first  version  of  the 
system.  In  production,  actual  cost  is  adjusted  for 
quantity  change,  by  means  of  a  price  improvement 
ciuve. 

Quantity  growth  is  defined  as  the 
percentage  by  which  actual  quantity  exceeds  planned 
quantity,  while  schedule  growth  is  defined  as  the 
I>ercentage  by  which  the  actual  schedule  exceeds  the 
plaimed  schedule.  The  development  schedule  is 
defined  as  the  number  of  months  from  Milestone  II 
to  IOC,  while  the  production  schedule  is  defined  as 
the  number  of  months  from  Milestone  III  to  the  end 
of  the  production  run  for  the  first  version  of  the 
system. 

Information  on  the  acquisition  program 
initiatives  applied  to  each  of  the  programs  was 
obtained  from  the  SARs  and  from  questionnaires 
submitted  to  the  program  managers.  The  measures  of 
effectiveness  were  then  compared  to  determine  the 
effectiveness  of  the  acquisition  program  initiatives. 
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n.  GROUND  COMBAT  PROGRAMS 

We  selected  nine  ground  combat  programs 
to  include  in  our  analysis:  three  surface-launched 


tactical  munitions  platforms,  one  surface-launched 
artillery,  three  armored  vehicles,  one  intelligence 
processing  system  platform,  and  one  field  artillery. 
The  programs  are  listed  in  Table  1. 


Table  1 .  Ground  Combat  Programs 


Proeram 

Type 

New  or 
Modification 

Producers 

FAADS  LOS-R 
(Avenger) 

Surface-Launched 
Tactical  Munition 

New 

1,207 

Boeing  Aerospace  Co. 

Sgt.  York  (DIVAD) 

Surface-Launched 
Tactical  Munition 

New 

64 

Ford  Aerospace  and  Comm 
Corp. 

MLRS 

Surface-Launched 
Tactical  Munition 

New 

4,813 

LTV  Aerospace  and  Defmse 
Company 

Roland 

Surface-Launched 
Tactical  Munition 

New 

27 

Hughes  Aircraft  Co.  and 
Boeing  Aerospace  Co. 

Ml  Abrams  Tank 

Armored  Vehicle 

New 

2,488 

General  Dynamics 

Bradley  FVS 

Armored  Vehicle 

New 

2,300 

FMC  Corp. 

M60A2 

Armored  Vehicle 

Mod 

543 

Chrysler  Corp. 

ASAS/ENSCE 

Intelligence  Processing 
System 

Mod 

N/A 

Jet  Propulsion  Laboratory 

M198  155mm  Towed 
Howitzer 

Field  Artilleiy 

New 

584 

Rock  Island  Arsenal 

Five  of  the  nine  systems  are  still  in 
production.  The  earliest  program  started  full-scale 
development  in  1970  and  began  production  in  1976 
(M198  lS5mm  towed  howitzer).  Two  armored 
vehicle  programs,  the  Ml  Abrams  tank  and  the 
Bradley  Fighting  Vehicle  System  (FVS),  started  full- 
scale  development  in  1976,  started  production  in 
1979  and  1980,  and  are  still  in  production.  These 
two  programs  underwent  two  and  three  major 
modifications  respectively  after  they  were  developed. 


We  included  two  programs  Uvit  were 
terminated  due  to  cost  and  performance  problems, 
Roland  and  Sgt.  York  (DIVAD).  Median 
development  time  for  ground  combat  programs  is  8S 
months.  The  production  time  span  has  a  median  of 
69  months,  including  the  projected  time  for 
programs  that  are  still  in  production.  Ground  combat 
programs'  development  and  production  start  dates, 
the  initial  operational  capability  (IOC)  and  projected 
production  end  dates  are  shown  in  Table  2. 


Table  2.  Development  and  Production  Times  for  Ground  Combat  Programs 


Program 

FSD  Start 

IOC 

Dev.  Time 
(in  months) 

Prod. 

Start 

Prod. 

End 

Prod,  lime 
(in  months) 

FAADS  LOS-R  (Avenger) 

Nov-86 

Sep-89 

34 

Aug-87 

Sep-97 

121 

Sgt.  York  (DIVAD) 

Nov-77 

Mar-87 

112 

May-82 

Scp-85 

40 

MLRS 

Jan-77 

Mar-83 

74 

May-80 

Sep-94 

172 

Roland 

Jan-75 

Jul-84 

114 

May-79 

Sep-81 

28 

M 1  Abrams  Tank 

Nov-76 

Jan-81 

50 

Apr-79 

Sep-83 

53 

Bradley  FVS 

Nov-76 

Dec-83 

85 

Feb-80 

Sep-84 

55 

M60A2 

Mar-65 

Sep-74 

114 

Mar-68 

Sep-73 

66 

ASAS/ENSCE 

Dec-84 

Apr-89 

52 

Mar-87 

Sep-06 

234 

M198,  155mm  Towed 

Dec-70 

Apr-79 

100 

Dec-76 

Sep-82 

69 

Howitzer 

Mean 

82 

97 

Standard  Deviation 

31 

69 
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In  this  section  we  describe  our  analyses  of 
ground  combat  programs  in  three  areas: 
distinguishing  features,  program  outcomes,  and 
effects  of  acquisition  initiatives  on  program 
outcomes. 

1.  Distinguishing  Features  of  Ground  Combat 
Acquisition  Programs 

Three  features  distinguish  the  ground 
combat  acquisition  programs  from  the  other 
acquisition  programs; 

1.  Larger  numbers  of  units  are  generally  produced 
relative  to  other  types  of  weapon  systems.  The 
lowest  quantity  to  be  produced  in  the  ground 
combat  programs  (excluding  the  two  canceled 
programs)  is  543. 

2.  The  use  of  accelerated  acquisition  strategies  in 
order  to  field  the  system  as  quickly  as  possible  led 
to  several  practices:  (a)  concurrent  development 
and  production  before  operational  and  evaluation 
test  completion  (Sgt.  York,  Roland,  M60A2),  (b) 
procurement  of  nondevelopmental  items  (FAADS 
LOS-R,  ASAS/ENSCE),  (c)  procurement  of  a 
system  with  limited  capability  configuration 


(ASAS),  (d)  limited  urgent  procurement  (ASAS). 
These  practices  contributed  to  high  growth  in  cost, 
delays  in  schedules,  reduction  in  performance 
levels  (Roland),  and  acquisition  of  systems  that 
could  not  meet  performance  requirements  without 
major  modifications  (Sgt.  York,  M60A2,  FAADS, 
ASAS/ENSCE). 

3.  On  average,  the  ground  combat  acquisition 
programs  have  higher  development  schedule  and 
development  and  production  cost  growth  than 
other  systems.  Most  of  the  ground  combat 
acquisition  programs  examined  here  took  place 
when  military  spending  was  in  an  upward  treud. 
Perhaps  the  cost  and  schedule  problems  in  this  set 
of  programs  occurred  because  industry  had  to  :  dd 
capacity  to  meet  demand.  Another  potential  rea^on 
for  the  problems  is  the  lack  of  commonality 
between  ground  combat  program  subsystems. 

2.  Outcomes  of  Ground  Combat  Acquisition 
Programs 

Measures  of  acquisition  program  outcor  .cs 
for  the  ground  combat  programs  are  shown  in  Taole 

3. 


Table  3.  Outcome  Measures  for  Ground  Combat  Programs 


Program 

Development 

Schedule  Growth 

Development 

Cost  Growth 

Production 

Cost  Growth 

Stretch 

Total  Program 

Cost  Growth 

FAADS  LOS-R 
(Avenger) 

0 

5 

N/A 

N/A 

N/A 

Sgt.  York  (DIVAD) 

15 

29 

216 

371 

203 

MLRS 

6 

2 

-12 

4 

-8 

Roland 

115 

52 

383 

187 

319 

Ml  Abrams  Tank 

-b 

54 

36 

-39 

40 

Bradley  FVS 

23 

216 

259 

-48 

250 

M60A2 

217 

28 

121 

-1 

115 

ASAS/ENSCE 

49 

49 

N/A 

N/A 

N/A 

Ml98  155mm  Towed 
Howitzer 

30 

35 

29 

2 

31 

The  Bradley  FVS  has  the  highest 
development  cost  growth  of  the  ground  combat 
programs.  That  is  probably  because  that  program 
inherited  its  engineering  development  and  funding 
from  the  troublesome  Mechanized  Infantry  Combat 
Vehicle  (MICV)  program,  which  was  terminated 
after  five  years  in  development.  TTie  MICV  problems 
included  failure  to  meet  system  weight  specifications 


(approximately  1,000  pounds  heavier),  and 
performance  requirements,  such  as  reliability  and 
durability,  which  resulted  in  design  changes,  hence, 
high  cost  growth. 

There  is  enormous  variability  in  production 
and  total  program  cost  growth  among  the  ground 
combat  programs.  M60A2  and  Roland  had 
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performance  problems  and  design  changes,  the  cost 
efifects  of  which  were  aggravated  by  concurrency  in 
development  and  production.  Both  Sgt.  York  and 
Bradley  failed  to  meet  performance  requirements, 
Sgt.  York  due  to  its  accelerated  acquisition  strategy, 
and  Bradl^  due  to  the  addition  of  more  advanced 
technology.  The  MLRS  program,  on  the  other  hand, 
actually  came  in  under  its  cost  estimate. 

Development  schedule  growth  for  ground 
combat  programs  is  high,  50  percent.  In  specific 
cases,  this  high  growth  was  due  to  concurrency  of 
system  development  and  production,  system  failure 
to  meet  performance  requirements  (e.g.,  M60A2  and 
Roland),  and  system  design  restructure  during  the 
development  phase  (e.g.,  M198  155mm  howitzer). 
Lower  production  schedule  growth  is  partly  due  to 
the  cancellation  of  two  programs  out  ( f  nine  in  the 
sample  (Roland  and  SgL  York)  after  only  three  years 
in  production.  Also  for  the  Bradley  FVS  and  the  Ml 
Abrams  Tank,  only  original  versions  are  considered 
for  the  analysis — that  is,  the  systems  n  production 
were  modified  to  a  new  version  befoie  production 
reached  the  quantity  plaimed.  Production  quantity 
growth  is  much  lower  for  the  same  reas  jn. 

3.  Effects  of  Acquisition  Initiatives  on  Ground 

Combat  Program  Outcomes 

Six  acquisition  initiatives  have  been  applied 
to  different  ground  combat  programs  in  various 
combinations.  These  initiatives  are; 

•  Prototyping— five  programs; 

•  Competition — three  programs  m  advanced 
development,  four  programs  in  full-scale 
development  (FSD),  no  programs  in 
production; 

•  Multi-year  procurement  fMYP)— two 
programs; 

•  Design-to-cost  (DTC)— four 

programs; 

•  Contract  incentives— four  in  FSD,  and 
three  in  production;  and 

•  Finn  fixed-price  development 
(FPD) — one  program. 

The  total  package  procxirement  initiative 
was  not  applied  to  any  ground  combat  program. 


Acquisition  initiatives  in  ground  combat  programs 
are  shown  in  Table  4. 

We  examined  Pearson  correlation 
coefficients  of  the  1/0  variables  representing  the 
acquisition  initiatives  with  the  outcome  measures. 
There  are  only  two  relationships  of  even  borderline 
statistical  significance:  MYP  is  negatively  correlated 
with  PCG  (significance  level=.ll)  and  TPCG  (.09), 
and  incentive  contracting  in  production  is  positively 
correlated  with  PCG  (.11)  and  TPCG  (.10).  In 
general,  aside  from  MYP,  initiatives  found  to  be 
effective  in  other  programs  do  not  seem  to  have  the 
same  effect  here.  To  some  extent,  this  may  be 
because  of  a  small  sample  size  and  the  unique 
problems  of  the  ground  combat  programs. 

In  addition,  the  variability  of  program 
outcomes  is  high.  When  we  examine  averages,  one 
program  such  as  Roland  may  skew  the  results.  When 
we  omit  the  two  programs  that  did  not  engage  in  any 
substantial  production  (Roland  and  Sgt.  York),  there 
are  no  significant  correlations  between  the  initiatives 
and  the  outcome  measures. 

Nc'-ertheless,  our  small  sample  size  allows 
us  to  look  n.ore  closely  at  specific  programs.  While 
the  relationjhips  revealed  by  doing  this  arc  not 
statistically  significant,  they  are  often  interesting. 

a.  Prototyping 

Development  cost  growth  in  prototyped 
ground  combat  programs  is  higher  than  in  non* 
prototyped  programs.  However,  production  cost 
growth  is  lower,  an  indication  that  prototyping  may 
have  some  beneficial  effect  (Figure  1).  The 
mechanism  for  this  benefit  appears  to  be  the 
following;  prototyping  results  in  less  development 
schedule  growth  (an  average  of  20  percent  for 
prototyped  vs.  48  percent  for  non-prototyped). 
Prototyped  programs  reach  IOC  closer  to  their 
plaimed  tim  .s.  We  have  seen  that  development 
schedule  growth  is  a  key  driver  of  cost  growth  in 
production,  'his  better  adherence  to  schedule  may 
help  programs  to  avoid  the  trap  of  technical  revision 
and  stretchout  that  have  plagued  ground  combat 
programs. 
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Table  4.  Acquisition  Initiatives  Applied  in  Ground  Combat  Programs 


Competition 

Multi-year 

Program 

Prototype 

Adv.  Devel. 

FSD 

Production 

Procurement 

FAADS  LOS-R  (Avenger) 

No 

Yes 

Yes 

No 

No 

Sgt.  York  (DIVAD) 

No 

No 

Yes 

No 

No 

MLRS 

Yes 

Yes 

Yes 

No 

Yes 

Roland 

No 

No 

No 

No 

No 

Ml  Abrams  Tank 

Yes 

No 

Yes 

No 

No 

Bradley  FVS 

Yes 

Yes 

No 

No 

No 

M60A2 

No 

No 

No 

No 

No 

ASAS/ENSCE 

Yes 

No 

No 

No 

No 

M198,  155mm  Towe 

Yes 

No 

No 

No 

Yes 

Howitzer 

Contract  Incentives 

Design- 

Fixed-Price 

Program 

to-Cost 

Dev. 

FSD 

Production 

FAADS  LOS-R  (Avenger) 

No 

No 

No 

No 

Sgt.  York  (DIVAD) 

Yes 

Yes 

No 

Yes 

h;LRS 

Yes 

No 

Yes 

No 

Roland 

No 

No 

Yes 

Yes 

Ml  Abrams  Tank 

Yes 

No 

Yes 

Yes 

Bradley  FVS 

Yes 

No 

No 

No 

N'60A2 

No 

No 

No 

No 

ASAS/ENSCE 

No 

No 

No 

No 

M198,  I55mm  Towe 

No 

No 

No 

No 

Howitzer 

OEvaonar  m>  nmcnoi  can  oimm  m  pwiunra  m 


Figurel .  Outcome  Measures  for  Programs 
With  and  Without  Prototyping 


b.  Competition 

Dual-sourcing  in  production  was  not 
applied  to  any  of  the  ground  combat  programs  we 


are  examining  here.  There  are,  however,  four 
programs — the  Ml,  FAADS  LOS-R,  MLRS,  and  the 
Sgt.  York — that  had  more  than  one  prime  contractor 
involved  in  the  full-scale  development  phase. 
Outcomes  for  these  programs  compare  generally 
favorably  with  those  that  did  not  have  competition  in 
FSD  (average  total  program  cost  growth  of  16 
percent  vs.  132  percent),  but  the  impact  is  not 
stati.*^  Jcally  significant. 

c.  Multi-year  Procurement 

Only  two  ground  combat  programs  in  our 
study — MLRS  and  the  Ml 98  howitzer — used  multi¬ 
year  procurement  (MYP).  The  Bradley  and  the  Ml 
had  multi-year  contracts,  but  they  were  applied  to 
later  versions  than  the  ones  examined  in  this  study. 
The  two  multi-year  programs  had  considerably  better 
production  outcomes  than  the  rest  of  the  sample.  The 
average  production  cost  growth  was  8  percent  for  the 
two  MYP  programs  vs.  139  percent  for  the  other 
ground  combat  programs.  As  with  other  equipment 
types,  the  multi-year  commitment  protected  the 
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programs  from  and  thus  helped  to  prevent 
production  cost  growth. 

d.  Design-to-Cost 

For  the  dcsign-to<ost  (DTC)  initiative,  we 
have  four  ground  combat  programs  with  DTC 
(MLRS,  Bradley  FVS,  Sgt.  York,  and  Ml  Abrams 
tank)  and  five  without.  DTC  seems  to  affect 
production  cost  growth,  although  the  relationship  is 
not  statistically  significant.  It  has  mixed  results  in 
controlling  cost  for  the  Army  programs;  successful 
for  the  MLRS  and  the  Ml,  unsuccessful  for  the 
Bradley.  For  the  MLRS,  DTC  was  implemented  in 
combination  with  other  initiatives  such  as 
prototyping,  competition  in  both  adv’anced 
development  and  FSD,  multi-year  procurement,  and 
contract  incentives.  For  the  Ml  (original  version), 
DTC  was  implemented  in  the  program  with 
prototyping,  competition  in  FSD,  and  contract 
incentives  in  both  FSD  and  production.  DTC  was 
claimed  by  both  program  offices  to  have  been  very 
effective  in  controlling  cost  and  serving  as  a  good 
design  tool.  For  the  Bradley  (original  version),  DTC 
was  applied  with  prototyping  and  competition  in 
advanced  development,  but  was  dropped  fiom  the 
program  as  it  progressed  due  to  the  product 
improvement  program.  The  Army  kept  improving 
the  system  with  additional  technologies,  which  led 
cost  to  rise  higher  than  DTC  goals.  Even  for  the 
original  version  of  the  Bradley,  DTC  goals  were  not 
met. 

e.  Firm  Fixed-Price  Developmen 

Only  one  ground  combat  program  m  ed  firm 
fixed-price  development  tFPD),  the  Sgt.  Y<  rk  gun 
s>stem.  The  weakness  of  FPD  in  the  Sg"..  York 
program  was  probably  due  both  to  difficulty  with  the 

T  able  5. 


technical  performance  of  the  system  and  to  the 
limited  resources  available  to  the  contractor.  While 
the  development  cost  outcome  for  the  Sgt.  York 
looks  good,  the  program  was  highly  concurrent,  and 
the  production  cost  growth  was  very  high. 
Development  problems  spilled  over  into  production. 

f.  Contract  Incentives 

The  results  on  contract  incentives  are 
skewed  by  the  presence  of  two  failed  programs.  The 
technical  shortcomings  of  the  Roland  and  the  Sgt. 
York  probably  cannot  be  blamed  on  incentive 
contracting.  The  only  other  program  with  incentive 
contracts  in  both  development  and  production  was 
the  Ml  tank,  and  its  results  are  about  in  the  middle 
of  the  other  ground  combat  programs.  Bradley  and 
MLRS  had  incentives  in  development  only — cost 
growth  wa>  high  for  Bradley  and  low  for  MLRS. 
From  this  evidence,  we  can  see  no  consistent  pattern 
of  impact  of  incentive  contracting  in  ground  combat 
programs. 

in.  SHIPS  AND  RELATED  NAVY  PROGRAMS 

Fifteen  ship  programs  were  the  focus  of  this 
portion  of  the  analysis;  three  combat  data  systems, 
ten  classes  of  surface  vessels,  and  two  classes  of 
attack  submarines.  The  programs  are  listed  in  Table 
S.  The  three  combat  data  systems  are  being  produced 
by  two  different  companies.  Nine  different  shipyards 
have  been  used  to  produce  the  ten  classes  of  surface 
ships,  and  two  different  shipyards  are  being  used  to 
produce  the  two  classes  of  submarines.  Four  of  the 
ship  classes  are  modifications  of  pre\ious  ship 
classes. 


ip  Programs 


Program 

Class 

Type 

New/Mod 

Quantity 

Producers 

AEGIS 

— 

Air  defense  /stem 

New 

65 

General  Electric  (RCA) 

AN/BSY-1 

— 

Submarine  combat 
system 

New 

NA 

IBM 

AN/BSY-2 

Submarine  combat 
system 

New 

31 

General  Electric 

SURTASS/ 

T-AGOS 

Stalwart 

Ocean  surveillance 

New 

18 

Tacoma  Boat  Building, 

Halter  Marine 

CG-47 

Ticonderoga 

Surface  combatant 

Mod  of 
DD-963 

27 

Bath  Iron  Works, 
Litton/Ingalls 

DD-963 

Spruance 

Surface  combatant 

New 

31 

Litton/Ingalls 

DDG-51 

Arleigh 

Burke 

Surface  pombatant 

New 

38 

Bath  Iron  Works, 
Litton/Ingalls 
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FFG-7 

Oliver  Haz¬ 
ard  Perry 

Surface  combatant 

LHA-i 

Tarawa 

Amphibious  warfare 

LHD-1 

Wasp 

Amphibious  warfare 

LSD-41 

Whidbcy 

Island 

Amphibious  warfare 

LSD-41CV 

Harpers 

Ferry 

Amphibious  warfare 

SSN-21 

Seawolf 

Attack  submarine 

SSN-688 

Los  Angeles 

Attack  subrruuine 

TAO-187 

Henry  J. 
Kaiser 

Replenishment  oiler 

New 

51 

Bath  Iron  Works,  Todd/ 
Seattle,  Todd/San  Pedro 

New 

5 

Litton/Ingalls 

Mod  of 
LHA 

6 

Litton/Ingalls 

Mod  of 
LSD-36 

8 

Lockheed,  Avondale 

Mod  of 
LSD-41 

6 

Avondale 

New 

9 

General  Dynamics/  Electric 
Boat,  Newport  News 

New 

62 

General  Dynamics/  Electric 
Boat,  Newport  News 

New 

18 

Avondale,  Perm  Shpbldg/ 
Tampa  Shpyd 

The  dates  each  of  the  systems  started  operational  capability  c  .tes  and  projected  production 

development  and  production,  along  with  the  initial  end  dates,  are  shown  ii  Table  6. 


Table  6.  Oevi  .opment  and  Production 
Start  and  End  Oates  for  Ship  Programs 


Program 

Class 

FSD 

Start 

Production 

Start 

IOC 

Production 

End 

AEGIS 

— 

12/69 

1/78 

9/83 

FYOl 

AN/BSY-l 

— 

9/83 

N/A 

3/90 

N/A 

AN/BSY-2 

— 

2/88 

12/95 

5/95 

FY99 

SURTASS/T-AGOS 

Stalwart 

10/74 

5/77 

8/83 

FY87 

CG^7 

Ticonderoga 

1/78 

1/78 

9/83 

FY94 

DD-963 

Spruance 

6/70 

6/72 

6/77 

2/83 

DDG-51 

Arleigh  Burke 

12/83 

10/86 

2/92 

FYOl 

FFG-7 

Oliver  Hazard  Perry 

12/83 

10/86 

2/92 

FYOl 

LHA-1 

Tarawa 

4/69 

1/71 

5/77 

FY81 

LHD-1 

Wasp 

7/82 

6/83 

10/90 

FY98 

LSD-41 

Whidbey  Island 

11/78 

1/81 

2/86 

FY92 

LSD-41CV 

Harpers  Ferry 

12/87 

11/89 

10/90 

FY99 

SSN-21 

Seawolf 

6/85 

6/88 

5/95 

FY99 

SSN-688 

Los  Angeles 

11/68 

1/71 

11/76 

FY97 

TAO-187 

Henry  J.  Kaiser 

12/81 

11/82 

2/87 

FY94 

Note:  N/A  means  that  data  were  either  not  available  or  insufhcient. 


1.  Distinguishing  Features  of  Ship  Acquisition 
Programs 

There  are  several  feamres  which  distinguish 
the  ship  acquisition  programs  from  the  other 
acquisition  programs.  The  first  is  that  there  are 
generally  low  numbers  of  units  produced  with  very 
high  unit  costs.  The  greatest  quantity  to  be  produced 
in  any  of  the  ship  programs  is  65,  and  the  median  for 
the  fourteen  programs  for  which  quantity  data  are 


available  is  29.  The  median  of  the  average  unit 
production  costs  for  the  ship  programs  is  over  $530 
million,  and  the  highest  of  the  average  unit 
production  costs  is  over  S 1 .45  billion. 

The  second  distinguishing  feature  of  the 
ship  acquisition  programs  is  that  development  costs 
are  a  low  proportion  of  total  program  costs.  Across 
the  twelve  programs  for  which  total  cost  data  are 
available,  the  mean  percentage  of  development  costs 
to  total  costs  is  3.3  percent,  and  the  median  is  an 
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even  lower  1.6  percent.  There  are  two  basic  reasons 
for  these  low  percentages.  The  first  is  that  much  of 
what  the  rest  of  the  defense  industry  refers  to  as 
development  costs  is  included  in  Navy  production 
costs;  in  particular,  the  costs  of  detailed  design  are 
typically  fluided  by  Ship  Construction  Navy 
appropriations  rather  than  Navy  development 
appropriations.  The  second  reason  is  that  the 
production  costs  for  the  ship  programs  include  the 
costs  not  only  of  the  ship  and  its  associated 
propulsion  (except  for  certain  nuclear  powerplant 
costs)  and  auxiliary  equipment,  but  also  the  costs  of 
the  combat  systems  with  which  the  ship  is  equipped. 

The  third  distinguishing  feature  of  the  ship 
acquisition  p  grams  is  that  they  have  be.  "’  taking 
place  at  a  t  le  of  great  overcapacity  in  he  U.S. 
shipbuilding  industry.  The  numbers  of  merchant 
ships  over  1,000  gross  tons  produced  annually  by 
I'.S.  shipyards  for  both  domestic  and  foreign  ship 
owners  have  declined  since  1949,  because  of  cheaper 
operating  costs  for  foreign  fiag  shipping,  and  a 
recurrent  boom  and  bust  cycle  of  overcapacity  in 
available  merchant  shipping  tonnage  world-wide. 
There  are  few  alternative  products  for  U.S.  shipyards 
to  produce,  other  than  ofiF-shore  oil  platforms,  and 
structural  and  equipment  assemblies  for  building 
construction  and  industrial  use.  As  a  result,  there  has 
been  short-run  pressure  on  shipyards  to  compete 
with  lower  prices.  In  the  longer  run,  shipyards  will 
go  out  of  business,  either  voluntarily  or  involuntarily 
as  in  the  cases  of  Lockheed  Shipbuilding  and 
Construction.  Pennsylvania  Shipbuilding,  and  Todd 


Shipyards,  among  others.  Less  overcapacity,  and  a 
smaller  number  of  competitors  in  the  future  may 
result  in  higher  production  costs  for  naval  ships. 

The  fourth  distinguishing  feature  of  the  ship 
acquisition  programs  is  the  high  cost  of  adapting 
equipment  to  operate  in  the  stringent  marine 
operating  environment.  The  two  primary 
environmental  problems  are  corrosion  from  saltwater 
and  humid  sea  air,  and  the  pv^unding  and  shocks  to 
the  hull  of  the  ship  as  it  moves  through  the  seas.  As 
a  result,  non-maritime  combat  equipment  is  rarely 
adapted  to  maritime  use  (though  the  RIM-7H  Sea 
Sparrow  surface-to-air  missile  and  the  BGM-109 
Tomahawk  cruise  missile  can  be  used  at  sea  and 
ashore).  Further,  the  costs  of  combat  equipment 
developed  for  the  maritime  environment  inhibits  its 
use  elsewhere.  As  a  result,  'I  ;re  is  an  incentive  to 
use  expensive  combat  sysi’ms  developed  for  a 
maritime  environment  ac:  oss  as  many  Navy 
programs  as  possible.  This  economizes  on  both 
development  and  production  >  osts. 

2.  Outcomes  of  Ship  Acquisition  Programs 

Acquisition  program  outcomes  for  the 
fifteen  ship  programs  are  shown  in  Table  7.  Unlike 
the  other  acquisition  programs  in  the  study, 
development  quantity  growth  is  not  included  among 
the  acquisition  measures  for  the  ship  programs, 
because  development  quantity  was  typically  not 
specified  for  the  ships.  (When  it  was  specified,  there 
was  no  change  from  the  specified  quantity.) 


The  only  ship  acquisition  program  outcome 
measure  that  has  a  significant  relationship  to  its  FSD 
year  is  development  schedule  growth.  Development 
schedule  growth  is  generally  higher  for  the  earlier 
programs  and  lower  for  the  later  programs,  and  this 
relationship  is  statistically  significant  at  the  2.5- 
percent  level.  This  would  suggest  that  recent 
acquisition  policies  have  been  effective  at  producing 
ships  on  schedule. 

For  the  ship  acquisition  programs, 
development  schedule  growth  is  not  related  to  either 
development  cost  growth  or  production  cost  growth. 
On  average,  production  cost  growth  is  higher  for 
f  '■ograms  with  higi  ;r  development  schedule  grov  Ji, 
hut  the  rclationshi  ■  is  not  statistically  significant 

However,  total  program  cost  growtli  is 
higher,  on  average,  for  programs  with  higher 
development  cost  growth,  and  that  relationship  is 
statistically  significant  at  the  10-percent  level. 


Problems  which  increase  the  proportionately  small 
development  costs  (small  in  relation  to  total  costs) 
are  likely  to  cany  over  into  the  detailed  design 
efforts  that  are  fimded  out  of  production  costs. 
Production  cost  growth  was  generally  less  for 
programs  that  were  modifications  of  previous  ship 
programs,  and  this  relationship  was  statistically 
significant  at  the  4-percent  level.  The  same  result 
had  been  observed  for  other  types  of  programs. 

3.  Effects  of  Acquisition  Initiatives  on  Ship 
Program  Outcome  Measures 

'fhe  major  iiiitiatives  applied  to  the  shii- 
programs  are  shown  in  Table  8.  We  examineu 
Pearson  correlation  coefficients  of  the  1/0  variables 
representing  the  acquisition  initiatives  with  the 
outcome  measures.  There  were  four  significant 
correlations; 


Table  8.  Acquisition  Initiatives  Applied  in  Ship  and  Related  Navy  Programs 


Program 

Dual-Sourcing 

FSD  Production 

Total 

package 

Procurement 

Multi-year 

Procurement 

AEGIS 

Yes 

Yes 

No 

No 

No 

AN/BSY-1 

Yes 

No 

No 

No 

No 

AN/BSY-2 

Yes 

Yes 

No 

No 

No 

SURTASS/T-AGOS 

Yes 

Yes 

Yes 

Yes 

Yes 

CG-47 

Yes 

No 

Yes 

No 

No 

DD-963 

Yes 

No 

No 

Yes 

No 

DDG-51 

No 

No 

Yes 

No 

No 

FFG-7 

Yes 

Yes 

Yes 

No 

No 

LHA-1 

No 

No 

No 

Yes 

No 

LHD-1 

No 

No 

Yes 

No 

No 

LSD-41 

No 

Yes 

Yes 

No 

No 

LSD-41CV 

No 

No 

Yes 

No 

Yes 

SSN-21 

Yes 

No 

Yes 

No 

No 

SSN-6R8 

No 

No 

Yes 

No 

Yes 

TAG- 137 

No 

No 

Yes 

No 

No 

Contract  Incentives 

Design- 

Fixed-Price 

Program 

to-Cost 

Dev. 

FSD 

Production 

AEGIS 

No 

No 

Yes 

Yes 

AN/BSY-1 

Yes 

No 

Yes 

Yes 

AN/BSY-2 

Yes 

No 

Yes 

Yes 

SURTASS/T-AGOS 

Yes 

No 

No 

Yes 

CG-47 

No 

No 

Yes 

Yes 

DD-963 

No 

No 

Yes 

Yes 

DDG-51 

Yes 

No 

N/A 

Yes 

FFG-7 

Yes 

Yes 

Yes 

LHA-1 

N/A 

Yes 

Yes 

LHD-1 

Yes 

Yes 

Yes 

LSD-41 

Yes 

Yes 

Yes 

LSD-41CV 

Yes 

No 

Yes 

SSN-21 

No 

No 

Yes 

SSN-688 

N/A 

N/A 

Yes 

TAO-187 

Yes 

Yes 

Yes 

*  Total  package  procurement  is  related 
to  higher  production  cost  growth  and  total 
program  cost  growth  (.OS).  Total  package 
procurement  was  also  related  to  higher 
development  schedule  growth  (.02). 

*  Prototyping  is  related  to  higher 
deve'opment  cost  growth  (.03),  contrary  to 
expA.  tations.  Perhaps  prototyping  is  more  difficult 
in  ship  programs.  In  some  sense,  the  lead  ^p 
serves  as  a  prototype— production  of  the  lead  ship 
is  where  problem-solving  takes  place.  Programs 
made  extensive  use  of  prototyping  at  the 
subsystem  level.  However,  it  may  be  that  we  are 
not  seeing  benefits  because  non-prototyped 
programs  used  an  even  more  cost-effective 
strategy— using  common,  already-developed 
subsystems.  For  example .  the  AN/SPS-49  air 
search  radar  was  used  by  five  of  the  ships  we 
examined-CG-47,  DD-963.  FFG-7,  LHD-I,  and 
LSD-41.  The  AN/SQQ-89  towed  array  sonar  was 
used  by  four  ships-CG-47,  DD-963,  DDG-51, 
and  FFG-7. 

*  Incentive  contracts  in  FSD  are  related 
to  lower  development  cost  growth  (.02). 

It  is  surprising  that  we  do  not  see  a 
statistically  significant  effect  of  dual-sourcing,  a 
major  Navy  initiative,  particularly  in  the  1980s. 
During  that  time,  dual-sourcing  was  virtually 
universal  for  the  programs  in  this  group.  We  have 
only  two  programs  with  production  data  that  were 


not  dual-sourced — the  LHA  and  the  DD-963.  Their 
production  cost  growth  is  higher  than  the  dual- 
sourced  programs. 

IV.  CONCLUSION 

Ground  combat  programs  include  military 
vehicles  and  a  set  of  distinctive  Army  programs.  On 
average,  the  total  program  cost  growth  for  ground 
combat  programs  is  substantially  higher  than  for 
other  equipment  types,  while  development  cost 
growth  and  development  schedule  growth  are  among 
the  highest.  The  averages  mask  considerable 
variability,  however,  and  study  of  individual 
programs  is  necessary  for  a  clear  understanding.  The 
Roland,  which  was  canceled,  has  the  highest  total 
program  cost  growth,  319  percent  The  MLRS 
program  actually  displayed  negative  cost  growth. 
The  Bradley  Fighting  Vehicle  System  has  the 
highest  development  cost  growth,  216  percent 
Development  schedule  growth  for  ground  combat 
programs  is  high,  SO  percent  on  average. 

The  outcome  measmes  for  ground  combat 
programs  are  very  different  from  the  outcome 
measures  for  otlier  types  of  acquisition  programs.  On 
average,  the  total  program  cost  growth  and 
production  cost  growth  for  the  ground  combat 
programs  are  substantially  higher  than  other  types, 
while  development  cost  growth  is  similar  to  the 
uctical  munitions.  For  comparison,  acquisition 
program  outcome  means  of  ground  combat  programs 
and  other  equipment  programs  are  shown  in  Table  9. 


Table  9.  Comparison  of  Ground  Combat  and  Ship  Outcomes  with  Other  Types  of  Equipment 


Equipment  Type 

N 

Programs 

(Dev.) 

N 

Programs 

(Prod.) 

Ave. 

Months  in 
Dev. 

Ships  and  Related 

14 

10 

90 

Ground  Combat 

9 

9 

82 

Aircraft 

27 

25 

72 

Tactical  Munitions 

30 

26 

82 

Other 

19 

14 

92 

Ave. 

Development 

Development 

Total  Program 

Months  in 

Schedule 

Cost  Growth 

Cost  Growth 

Prod. 

Growth 

(ave.) 

(ave.) 

_ _ 

166 

17% 

23% 

18% 

97 

50 

52 

136 

135 

14 

33 

27 

119 

50 

56 

68 

119 

41 

55 

42 

Note:  Total  program  cost  growth  could  be  calculated  for  only  seven  of  the  nine  ground  combat  programs. 
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We  examined  whether  management 
initiatives  result  in  more  favorable  outcomes  for 
ground  combat  programs.  We  found  that,  when  we 
exclude  the  two  programs  that  did  not  have  any 
substantial  production,  there  are  few  statistically 
significant  correlations  between  the  initiatives  and 
the  outcome  measures.  There  are,  however,  some 
interesting  relationships.  For  example,  production 
cost  growth  averaged  8  percent  for  the  MLRS  and 
the  Ml 98  howitzer,  which  had  multi-year  contracts, 
while  average  PCG  was  139  percent  for  programs 
without  multi-year  contracts. 

The  use  of  accelerated  acquisition  strategies 
in  ground  combat  programs  fiequently  led  to 
unfavorable  cost  and  schedule  outcomes.  In  addition, 
these  practices  contributed  to  performance  problems. 
In  several  cases,  the  system  had  to  be  abandoned  or 
modified.  Other  factors  that  may  have  contributed  to 
problems  in  ground  combat  programs  are  lack  of 
industry  capacity  and  lack  of  subsystem 
conunonality. 

The  Navy  ships  and  related  acquisition 
programs  comprise  three  combat  data  systems,  ten 
classes  of  surface  vessels,  and  two  classes  of  attack 
submarines.  Total  program  cost  growth  (18  percent, 
on  average)  is  considerably  less  for  ships  and  related 
programs,  when  compared  with  the  other  equipment 
types.  The  average  of  23  percent  development  cost 
growth  is  also  less  than  for  other  equipment  types. 
Development  schedule  growth  averages  only  17 
percent — apparently,  ship  programs  had  a  good 
record  in  reaching  initial  operational  capability  when 
planned. 

The  major  initiatives  had  some  significant 
impact  on  program  outcomes.  Total  package 
procurement  is  related  to  higher  production  cost 
growth  and  total  program  cost  growth.  Prototyping  is 
related  to  higher  development  cost  growth,  contrary 
to  expectations.  Incentive  contracts  in  FSO  are 
related  to  lower  development  cost  growth. 

It  is  surprising  that  we  do  not  see  a 
statistically  significant  effect  of  dual-sourcing,  a 
major  Navy  initiative.  However,  we  have  only  two 
programs  with  production  data  that  were  not  dual- 
sourced,  and  that  may  be  a  factor  in  the  overall  low 
cost  growth  for  ships. 

These  outcome  measures  appear  to  be  quite 
different  from  the  outcome  measures  for  the  other 
acquisition  programs  in  the  study,  as  shown  in  Table 


9.  Total  cost  growth  and  production  cost  growth  are 
much  less  for  the  ship  programs,  while  development 
cost  growth  is  somewhat  less  for  the  ship  programs. 
Development  schedule  growth  is  much  less  for  the 
ship  programs,  but  production  schedule  growth  is 
somewhat  less. 


Several  features  distinguish  the  ship 
acquisition  programs  from  the  other  acquisition 
programs.  The  first  is  that  there  are  generally  low 
numbers  of  units  produced  with  very  high  unit  costs. 
The  greatest  quantity  to  be  produced  in  any  of  the 
ship  programs  is  65. 

The  second  feature  is  that  development 
costs  :'re  a  low  proportion  of  total  program  costs.  (>n 
average,  development  costs  represent  less  than  S 
percent  of  total  cost.  Much  of  what  the  rest  of  the 
defense  industry  refers  to  as  development  costs  are 
included  in  Navy  production  costs,  and  productins 
costs  also  include  the  costs  of  the  combat  systeris 
with  which  the  ship  is  equipped. 

The  third  feature  is  that  ship  programs  ha  e 
been  pursued  at  a  time  of  great  overcapacity  in  tie 
U.S.  shipbuilding  industry.  As  a  result,  there  has 
been  great  pressure  on  shipyards  to  compete  with 
lower  prices. 

The  final  distinguishing  feature  is  the  high 
cost  of  adapting  equipment  to  operate  in  the  marine 
environment.  Protecting  ships  against  conosion  from 
saltwater  and  sea  air,  and  the  pounding  and  shocks 
to  the  hull  of  a  ship  from  normal  operations,  is  ve.y 
costly.  The  high  cost  of  adapting  to  the  marir.e 
environment  encourages  subsystem  commonality, 
which  may  be  a  significant  factor  in  the  low  cost 
growth  for  ship  programs. 
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ACQUISITION  MANAGEMENT  OF  CONSUMABLE  TECHNOLOGY 
UTILIZING  A  NOVEL  TECHNOLOGY  TRANSITION  PROCESS 


Russell  E.  Urzi  ,  USAF 


ABSTRACT 

The  traditional  approach  towards  acquisition 
of  military  hardware  is  inherently  slow.  By 
the  time  the  weapon  system  is  being 
delivered  many  of  it’s  subsystems  no  longer 
meet  the  real  needs  of  operational  missions. 
Also,  missions  themselves  evolve  and 
yesterdays  subsystem  may  no  longer  be 
adequate.  Hence,  the  system  is  forced  into 
expensive  retrofit  in  order  to  get  it  up  to  the 
desired  capability.  A  classic  example  of  this 
occurs  with  the  aircrew  transparency 
subsystem.  Typically,  the  aircrew 
transparency  subsystem  is  of  secondary 
concern  in  the  aircraft  design  process  and  in 
the  evolution  of  aircraft  mission  assignments 
during  the  life  of  the  aircraft.  This  can 
result  in  an  urgency  driven,  very  expensive, 
redesign,  and  in  this  current  tight  fiscal 
environment,  this  process  will  clearly  not  do 
anymore. 

A  proven  process  for  accelerated  transition 
of  technologies  can  be  used  to  overcome 
these  problems.  This  process  to  rapidly 
transition  laboratory  R&D  to  operational  use 
evolved  using  principals  which  are  now 
more  commonly  referred  to  as  a  TQM 
approach.  This  process  can  be  used  on  any 
subsystem  which  is  consumable,  that  is,  it 
wears  out  and  requires  periodic  replacement. 

This  paper  explains  this  process,  describes 
several  examples  of  its  application,  and 
suggests  applications  in  which  it  can  be  used 
in  a  broader  sense. 


INTRODUCTION 

A  common  criticism  is  that  the  results  of 
Laboratory  Research  and  Development 
efforts  are  too  slowly  exploited  into 
better/cheaper  operational  hardware.  Rapid 
advancements  are  being  made  in  the  areas  of 
cost  reduction  technology  and  to  achieve 
technological  superiority.  These  technology 
advancements  are  keeping  pace  with  the 
ever  changing  missions  demanded  of  the 
military. 

Because  some  of  these  technology 
advancements  have  only  recently  been 
available  and  the  rate  at  which  the 
laboratory  is  progressing  is  much  faster  than 
before,  some  subsystems,  while  acceptable, 
may  not  be  optimal.  For  their  time,  these 
subsystems  were  the  best  they  could  be,  but 
now  they  could  benefit  from  further 
technological  advances.  It  is  these 
subsystems  that  could  benefit  from  the 
proposed  process  to  rapidly  transition  the 
Research  and  Development  advancements  to 
operational  usage. 

To  illustrate,  take  for  example  the  evolution 
from  high  altitude  bombing  to  low  altitude 
precision  strikes.  Among  the  various 
aircraft  subsystems  which  must  also  evolve 
concurrently  is  the  aircraft  transparency. 
At  lower  altitudes,  there  is  an  increased  risk 
of  birdstrikes,  as  well  as  erosion  from 
particulate  matter  suspended  in  air.  The 
transparency  system  must  be  able  to  cope 
with  this  new  environment.  The 
transparency  system  of  an  existing  fighter 
aircraft  is  a  good  example  of  this  mission 
evolution  and  its  resulting  consequences. 
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An  early  transparency  system  (canopy)  was 
constructed  from  a  single  sheet  of  material 
and  had  a  birdstrike  resistance  of  less  than 
200  knots.  It  soon  became  apparent  that  at 
lower  altitudes  (below  3000  ft)  an  increase 
m  birdstrike  resistance  was  needed.  A 
canopy  was  developed  and  provided 
protection  to  350  knots.  However  more 
problems  developed  because  of  this  low 
level  environment,  chiefly,  abrasion  of  the 
canopy  resulting  in  a  loss  of  optical  clarity. 
(Operation  Desert  Storm  was  a  good 
example  of  this  phenomenon  being 
experienced  on  many  aircraft.)  Aircraft 
were  also  performing  this  mission  at  speeds 
higher  than  350  knots,  and  with  onboard 
equipment  necessitating  improved  optical 
quality  of  the  canopy.  Consequently,  the 
canopy  had  to  provide  even  more  birdstrike 
protection  as  well  as  abrasion  resistance  and 
improved  optics.  This  requirement 
evolution  took  place  over  a  15  year  period 
during  which  more  complicated  canopies 
were  developed  to  meet  these  requirements 
as  well  as  extending  the  service  life  of  these 
parts.  This  was  all  accomplished  by  using 
the  following  technology  transition  process. 

THE  PROCESS  -  "PRELUDE" 

The  process  can  be  explained  much  like  the 
tires  on  your  car.  They  wear  out  and  need 
to  be  replaced.  Your  type  of  car  and  to 
some  degree  your  personality,  will 
determine  what  type  of  tires  you  will 
purchase.  All  must  be  durable  and  low 
cost.  The  sports  car  driver  will  insist  upon 
good  traction  and  manuerverablity  for 
negotiating  curves,  the  off-road  four  wheel 
drive  vehicle  will  opt  for  more  rugged 
construction  and  better  traction,  and  finally 
the  land  yacht  owner  will  opt  for  quiet  and 
comfort.  Each  will  evaluate  the 
performance  and  durability  of  their  current 
tire  against  these  standards  in  selecting  a 
new  tire. 


The  tire  manufacturers  are  constantly 
striving  to  provide  better  products  in  order 
to  obtain  more  of  the  market  share  and 
consequently  more  profit.  Hence,  the 
consumer  (driver)  is  faced  with  many  more 
possibilities  than  say  a  year  or  two  ago.  He 
may  consult  with  a  auto  mechanic  or  test 
agency  before  making  any  purchase 
decisions.  Once  these  evaluations  are 
completed,  the  consumer  will  make  his 
choice  of  tires.  He  will  drive  these  tires, 
form  his  own  impressions  about  their 
performance  and  durability,  and  eventually 
wear  them  out.  The  process  then  starts  all 
over  again. 

This  oversimplified  explanation  illustrates 
the  key  concepts  involved.  One  fundamental 
concept  is  that  some  weapon  subsystems  are 
of  a  consumable  nature  and  eventually  wear 
out.  And,  that  when  they  do  wear  out,  the 
replacement  should  meet  the  requirements 
necessary  to  fulfill  the  mission. 

The  operational  users,  together  with  the 
system  developers,  maintainers,  industry, 
testing  and  supporting  laboratories  form  the 
basis  in  which  this  process  works  (i.e.  the 
stakeholders.) 

The  second  fundamental  concept  is  that  the 
stakeholders  who  have  ownership  of  the 
weapon  system  want  to  deliver  the  best 
product  they  can  possibly  achieve.  This  is 
absolutely  necessary  for  success!  For 
without  this  personal  commitment,  the 
weapon  system  is  doomed  to  mediocrity. 

When  these  two  fundamental  concepts  or 
conditions  are  met  or  exist,  then  the  process 
may  proceed  as  explained  in  the  next 
section. 
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THE  PROCESS  -  "FUGUE" 

The  first  step  in  the  process  is  to  identify 
what  performance  and/or  supportability 
features  of  the  subsystem  should  be 
improved.  The  subsystem  must  be  of  a 
consumable  nature  and  would  be  replaced 
periodically.  Usually  a  standard  criterion  is 
used  in  order  to  make  the  determination  of 
whether  the  current  system  is  good  enough 
or  should  be  improved.  However,  this 
standard  is  of  a  dynamic  nature  and  changes 
with  the  situation.  To  continue  the 
transparency  example,  the  Air  Force  uses 
the  "444"  concept  as  the  standard  to  which 
transparency  systems  are  compared. 
Simply  stated,  a  transparency  system  should 
perform  its  mission  for  a  minimum  of  four 
years  after  which  it  should  be  able  to  be 
replaced  by  no  more  than  four  technicians  in 
four  hours.  Embedded  in  this  standard  are 
both  performance  and  supportability 
features. 

Once  the  standard  is  identified,  a  decision 
must  be  made.  That  is,  should  the 
subsystem  be  upgraded,  procured  as  is,  or 
both?  Usually,  the  subsystem  being  of  a 
consumable  nature  can  be  upgraded,  but  the 
upgrade  must  be  proven  sufficient  to  warrant 
the  risk  of  purchase. 

This  decision  is  best  done  by  the 
stakeholders.  The  using  commands  will 
comment  on  the  performance,  the 
maintainers  on  the  supportability,  etc.  If  the 
current  subsystem  is  not  okay,  then  the 
stakeholders  must  proceed  to  the  next  step. 

The  next  step  relies  upon  the  supporting 
laboratory  to  apply  emerging  technology  to 
the  subsystem.  The  laboratory  is  constantly 
developing,  integrating,  validating  and 
transitioning  technology  to  meet  mission 
performance  requirements  at  lower  cost. 
Depending  upon  the  maturity  of  the 


technology,  the  subsystem  may  benefit 
immediately.  It  is  important  to  note  that  of 
the  stakeholders,  the  laboratory  is  best 
suited  for  this  task,  and  not  say  the  system 
developer.  The  reason  for  this  is  that  quite 
simply  the  laboratory  is  where  technology 
risks  are  taken  at  minimal  cost.  Taking  new 
technology  and  inserting  it  into  the 
Engineering,  Manufacturing  and 
Development  phase  is  much  riskier  and 
more  costly. 

When  the  new  technology  is  applied  to  the 
subsystem,  a  prototype  must  be  evaluated. 
This  evaluation  could  be  in  the  form  of  a 
"critical  experiment"  or  an  "operational 
evaluation".  What  is  important  in  this  step 
is  that  the  evaluation  must  include  all  the 
stakeholders.  If  for  example  the  operational 
command  was  not  present,  then  there  could 
not  be  any  feedback  on  it’s  suitability  for 
field  use.  Likewise,  a  similar  logic  could  be 
said  of  the  logistics  command.  The  basic 
point  is  that  the  ev  aluation  must  not  occur 
"in  a  vacuum". 

If  the  operational  evaluation  was  a  success, 
then  the  technologies  embodied  in  the 
subsystem  could  immediately  be  procured  in 
an  upcoming  spares  buy.  Again,  if  the 
stakeholders  were  properly  working  together 
from  the  beginning  this  would  not  pose  a 
logistics  problem.  If  the  subsystem  did  not 
perform  as  desired,  then  back  to  the 
laboratory  it  goes  for  continued  technology 
development.  This  ends  up  being  a 
somewhat  cyclic  process  paced  to  coincide 
with  specific  upcoming  spares  buys. 

As  a  case  example,  let's  examine  the 
transparency  subsystem  for  an  existing 
bomber.  The  current  production  design 
windshield  had  a  service  life  of  less  than  one 
year  (350  flight  hours)  due  to  delamination 
of  the  outer  glass  ply,  forcing  the  Air  Force 
to  spend  $10  million  per  year  for  spares. 


Addressing  this  service  life  issue  was  truly 
a  joint  effort  between  all  the  stakeholders. 
The  stakeholders  working  together  agreed  to 
a  service  life  interim  goal  of  two  years. 
Doubling  the  service  life  to  two  years  was 
definitely  not  consistent  with  the  first  "4"  (4 
year  service  life)  in  the  "444"  concept.  But, 
the  stakeholders  agreed  that  the  technologies 
to  support  an  interim  goal  of  two  years 
could  be  demonstrated,  with  reasonable 
risks,  in  time  to  coincide  with  an  upcoming 
spares  buy.  And,  that  the  savings  from  this 
procurement  against  the  interim  goal  would 
be  a  welcome  reduction  to  Air  Force  costs 
of  ownership  while  the  four  year  service  life 
goal  was  pursued. 

Technology  transition  to  users  was  realized 
as  the  prototype  windshield  designs  went 
into  production.  This  stakeholder  technology 
exploitation  team  was  working  so  well  that 
in  less  than  one  year  the  stakeholders  again 
selected  a  prototype  windshield  design  to 
replace  the  current  production  windshield. 


The  new  windshield  designs  provide  100% 
increased  service  life  at  half  the  cost  of  the 
initial  production  design.  The  Air  Force  is 
expected  to  save  $7.5M  per  year  for  spare 
parts  as  a  result  of  this  technology 
transition.  This  savings  is  coming  from 
increased  service  life  and  a  result  of 
competition  between  the  vendors.  The  unit 
cost  of  the  windshield  dropped  from  $62K  to 
$25-$33K  since  the  program  started. 

The  key  reason  for  this  success,  is  the  close 
relationship  that  has  been  developed  and 
maintained  between  all  the  stakeholders. 
The  users,  the  logistics  community,  the 
laboratory  and  industry  were  full  team 
members  from  the  earliest  phase,  and  their 
active  participation  throughout  the  program 
ensured  a  smooth  and  rapid  transitioning  of 


technology  as  soon  as  it  was  successfully 
demonstrated.  The  team  is  continuing  in  its 
quest  to  transition  emerging  technologies  to 
meet  the  "444"  goal. 

THE  PROCESS  -  "VARIATIONS" 

We  have  encountered  many  applications 
where  this  process  could  be  used  to  focus 
technology  development  and  then  rapidly 
transition  it  to  operational  use.  One  area 
which  is  just  beginning  to  use  this  process  is 
aircraft  engines.  The  Propulsion  System 
Program  Office  at  Wright-Patterson  uses  a 
similar  process  to  replace  engines  in  high 
performance  aircraft.  The  only  criteria  for 
successful  application  is  that  the  two 
fundamental  concepts  discussed  earlier  be  in 
place.  Other  examples  include  tires,  brakes, 
wheels,  and  ejection  seats.  This  process  is 
not  just  confined  to  subsystems  associated 
with  aircraft  but  could  be  applied  to  any 
weapon  system  or  platform. 

SUMMARY 

This  paper  has  explained  a  technology 
transition  process  to  rapidly  transition 
technology  from  the  laboratory  to 
operational  use.  The  only  criterion  needed 
is  that  the  subsystem  be  of  a  consumable 
nature  and  the  two  fundamental  concepts  be 
in  place.  Additioiially,  this  process  requires 
a  "team  approach"  to  accomplish  the 
transition.  This  is  probably  the  most  critical 
element  in  the  process,  because  if  this 
teaming  did  not  occur  then  technology 
transition  would  be  just  "over  the  wall". 
This  team  is  actually  the  stakeholders  in  the 
weapon  system.  They  have  a  personal 
commitment  to  producing/maintaining  the 
best  system  for  the  operational  commands. 
Together,  a  close  working  relationship 
among  the  stakeholders  is  the  key  to 
successful  technology  transition. 


636 


ENDNOTES 


1.  Draft  Proposed  Aeronautical  Systems 
Center  (ASC)  Technical  Planning  Integrated 
Product  Team  Charter  dated  4  Dec  92.  This 
charter  brings  together  the  vital  stakeholders 
in  the  systems  acquisition  process  in  a  forum 
to  meet  projected  operational  user  needs. 
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ABSTRACT 


During  Operation  Desert  Shield/Desert  Storm,  Department  of  Defense  (DOD)  contracting 
organizations  within  the  continental  United  States  were  called  upon  to  react  quickly  to 
defense  weapon  and  logistical  wartime  needs.  This  quick  reaction  required  extraordinary 
measures.  Contracting  professionals  for  all  the  military  services  and  the  defense  agencies 
were  required  to  research  the  regulations  and  public  laws  to  determine  what  was  permitted 
under  such  circumstances,  coordinate  with  other  levels  in  their  organizations  and  work 
closely  with  the  industrial  base.  Following  Desert  Storm,  much  effort  was  spent  definitizing 
and  documenting  wartime  contracting  efforts.  This  study  was  conducted  in  order  to 
determine  the  methods  used  by  the  military  services  and  Department  of  Defense  (DOD) 
agencies  to  contractually  procure  equipment,  supplies,  and  services  during  times  in  which 
procurement  by  normal  contracting  channels  would  endanger  national  strategies.  The  study 
centered  on  the  expeditious  transactions  that  those  agencies  located  within  the  continental 
United  States  (CONUS)  used  in  order  to  shorten  overall  procurement  times  for  the  equipment 
and  supplies  required  by  Operation  Desert  Shield  and  Operation  Desert  Storm  (ODS). 

INTRODUCTION 


Operation  Desert  Shield  refers  to  the  period 
from  August  7,  1990,  when  President  Bush 
ordered  U.S.  forces  to  Saudi  Arabia,  to 
January  17  1991,  when  the  coalition  air 
campaign  commenced.  Operation  Desert 
Storm  refers  to  the  period  from  January  17, 
1991  to  February  27,  1991  when  the  cease 
fire  was  declared.  It  is  important  to  note 
that  during  Operation  Desert  Shield,  the 
U.S.  had  approximately  four  months  to 
procure  necessary  supplies  before  the  air  or 
land  campaigns  began. 

This  particular  type  of  contracting,  best 
represented  by  the  phrase  "CONUS 
contracting,"  differs  from  contingency 
contracting  only  in  the  fact  that  the  needed 
supplies  are  purchased  in  CONUS  rather 


than  in  the  theater  of  operations. 

On  February  21,  1991,  Secretary  of  Defense 
Dick  Cheney  revealed  a  new  military 
strategy  in  an  address  to  the  Senate  Armed 
Services  Committee.  The  following 
quotation,  taken  directly  from  that  speech, 
best  describes  the  new  strategy: 

"The  most  important  change  reflected  in  this 
new  strategy  is  that  we  no  longer  are 
focused  on  the  threat  of  a  Soviet-led, 
European  wide  conflict  leading  to  global 
war... The  new  strategy  shifts  its  focus  to 
regional  threats  and  the  related  requirements 
for  forward  presence  and  crisis 
response... The  regional  contingencies  we 
might  face  are  many  and  varied... One  trait 
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most  of  them  share,  however,  is  that  they 
will  arise  on  very  short  notice  and  therefore 
require  a  highly  responsive  military 
capability."  (1) 

This  new  strategy  requires  the  United  States 
DOD  to  be  capable  of  the  rapid  deployment 
and  support  of  troops  anywhere  in  the 
world.  In  order  to  implement  this  strategy 
the  various  contracting  organizations  of 
DOD  must  possess  the  ability  to  award 
contracts  quickly  to  support  operations  in 
the  theaters  of  conflict. 

Operations  Desert  Shield  and  Desert  Storm 
are  examples  of  the  types  of  conflict  the 
military  may  experience  in  the  future.  All 
of  the  agencies  within  the  DOD  prepared  for 
ODS  in  their  own  ways.  The  preparations 
for  the  conflict  required  that  the  contracting 
professionals  research  regulations  and  public 
laws,  coordinate  with  other  military 
organizations,  and  work  closely  with  the 
defense  industrial  base  in  order  to  meet  the 
logistical  needs  of  the  troops  deployed  in  the 
theater  of  conflict  as  rapidly  as  possible. 
Consolidation  of  the  various  methods  used 
within  the  DOD  to  contractually  support 
ODS  has  the  potential  to  aid  in  educating 
contracting  professionals  to  effectively  and 
efficiently  support  a  future  military 
contingency.  Another  important  aspect  of 
this  study  is  to  reveal  any  particular 
regulations  and  laws  that  need  to  be  changed 
in  order  to  expedite  the  support  of  military 
contingencies. 

The  objective  of  this  research  was  to 
compile  success  stories,  lessons  learned,  and 
develop  a  quick  reaction  contracting  model. 
If  similar  conflicts  occur  in  the  future, 
defense  contractors  and  contracting 
organizations  will  be  called  upon  again  to 
quickly  react  again  to  weapon  and  logistical 
wartime  needs.  That  is  why  it  is  so  critical 
to  capture  and  document  the  lessons  the 


contracting  community  learned  during 
Operation  Desert  Shield/Desert  Storm. 

METHODOLOGY 

A  qualitative  research  study  was  conducted 
during  the  period  from  December  1991  to 
September  1992.  Focus  group  meetings 
were  conducted  at  organizations  which  were 
heavily  involved  in  providing  contracting 
support  during  ODS.  (2)  Personal 
interviews  were  also  conducted  with 
individuals  who  responded  to  press  releases 
about  the  research  and  requests  for 
information  published  in  journals.  During 
these  focus  group  meetings  and  interviews, 
open  ended  questions  were  asked  concerning 
experiences  in  providing  contracting  support 
during  ODS. 

Critical  factors  which  affected  contracting 
support  of  ODS  were  identified  through 
these  discussions.  The  researchers  then 
used  these  critical  factors  to  structure  a 
Delphi  questionnaire. (3)  Following  a  pre¬ 
test  for  content  validity  and  clarity,  the 
questionnaire  was  sent  to  a  select  group  of 
experts.  Initial  responses  were  compiled 
and  returned  to  the  same  group  of  experts 
for  further  comment.  Anonymity  of 
respondents  was  maintained  during  the  entire 
process. 

The  icsiilt  was  a  rich  body  of  information 
about  the  experiences,  lessons  learned,  and 
recommendations  of  people  at  the  grass 
roots  level  who  were  personally  involved  in 
providing  support.  The  respondents 
represented  all  the  military  services  and  key 
defense  agencies.  There  was  representation 
from  people  at  both  the  working  level  and 
senior  management.  In  order  to  assess  the 
possibility  of  meaningful  differences  among 
the  diverse  respondents  and  in  an  attempt  to 
add  additional  rigor  to  this  qualitative  study, 
a  matrix  was  developed  to  analyze  the  data. 
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The  data  was  examined  to  ascertain  whether 
meaningful  differences  existed  both  intra  and 
inter  organizationally.  Although  most  of  the 
respondents  were  involved  in  contracting 
positions,  there  also  was  representation  from 
program  management,  transportation,  and 
production  specialists. 

FINDINGS 

An  eighty  percent  response  rate  to  the 
Delphi  questionnaires  was  experienced. (4) 
The  information  provided  by  the  respondents 
to  the  study  was  generally  uniform  across 
the  services  and  DOD  agencies.  No  major 
differences  were  perceived  in  any  of  the 
answers  due  to  agency  affiliation. 

Differences  Among  Groups.  The  only 
difference  in  responses  among  groups  was 
attributable  to  differences  in  organization 
levels.  Respondents  in  senior  leadership 
positions  perceived  their  headquarters 
personnel  as  demonstrating  a  distinct  sense 
of  urgency  during  ODS.  Respondents  at 
lower  levels  in  the  organizations  held  the 
opposite  perception.  In  fact,  field  level 
personnel  cited  several  specific  instances  in 
which  they  felt  that  headquarters  personnel 
were  impediments  to  getting  needed 
products  to  users  -  either  through  inaction  or 
through  additional  requirements  imposed  on 
the  normal  processes. 

Below  is  a  summary  of  the  responses: 

ODS  Supplies  and  Services.  The 
respondents  were  involved  in  the  acquisition 
of  a  wide  variety  of  supplies  and  services 
for  ODS.  Among  the  items  they  procured 
were:  spare  parts,  equipment,  weapons, 
ammunition,  communications,  transport 
services,  food,  clothing,  aircraft 
modifications,  diagnosis  and  repair. 

Where  Used.  The  items  procured  by  the 


respondents  were  used  in  many  different 
areas.  Some  were  procured  for  use  in  the 
area  of  operations.  Others  were  to  remain 
in  the  U.S.  as  replacements  for  items 
already  shipped  to  the  theater  of  operations. 
Some  were  used  for  training,  for  repair 
depots,  and  as  reserves  worldwide. 

Contracting  Methods.  The  respondents  used 
a  variety  of  contracting  methods.  Where 
possible,  they  accelerated  delivery  on 
existing  contracts.  New  contracts  and 
purchase  orders  were  awarded,  options  were 
exercised,  orders  were  placed  against  Basic 
Ordering  Agreements  (BOA’s)  and 
requirements  contracts,  and  equipment  was 
leased.  The  "borrow-payback"  method  of 
taking  delivery  of  spare  parts  from  weapon 
system  production  lines  was  used.  Only  ten 
percent  of  the  respondents  used  cost  type 
contracts. 

Competition  was  achieved  for  a  large 
pumher  of  the  procurements;  however,  it 
was  not  always  "full  and  open"  competition 
due  to  the  urgency  of  the  situation.  Most  of 
the  procurements  were  negotiated;  placing 
an  enormous  workload  on  the  CONUS 
contracting  organizations.  Several 
respondents  reported  that  contractors  started 
work  without  contracts  -  at  their  own  risk. 

Source  Development.  Fifty  percent  of  the 
respondents  reported  that  it  was  necessary  to 
develop  new  sources  for  the  ODS 
procurements.  Various  methods  were  used 
for  source  development.  Several 
commercially  available  items  were  used; 
often  modification  to  meet  military  needs 
was  necessary. 

Accelerated  Delivery.  The  majority  of  the 
respondents  reported  specific  actions  taken 
to  accelerate  delivery.  Where  possible, 
inspections  were  accelerated  and  packaging 
specifications  were  waived  in  favor  of  "best 
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commercial"  practice.  In  some  cases 
contract  modific?tions  were  made  to  adjust 
the  delivery  schedules  of  items  not 
accelerated  as  a  trade-off.  One  respondent 
indicated  that  first  article  tests  were  waived 
in  order  to  facilitate  early  delivery. 

Approximately  half  of  the  respondents 
reported  that  there  was  an  additional  cost  to 
accelerate  delivery.  In  most  cases  the 
additional  costs  were  for  contractor 
overtime.  One  respondent  replied  that  the 
contractor  requested  premium  pay  and  the 
modification  of  delinquent  contracts  but  that 
the  government  refused  to  pay  for  those 
increases  in  price.  A  few  respondents 
reported  that  no  extra  charges  were  required 
to  accelerate  delivery. 

Relationships  With  Industry.  Most 
respondents  reported  very  good  relationships 
with  industry  during  ODS.  Many  anecdotes 
were  reported  of  contractors  "going  the 
extra  mile"  to  provide  assistance.  Some 
contractor  technicians  deployed  with  their 
systems  and  lived  in  tents  along  with  the 
military  members  using  the  systems. 
Detroit  Diesel  put  aside  their  commercial 
work  and  delivered  engines  in  two  weeks. 
Employees  of  Donaldson  Filters  showed  up 
early  at  the  plant  "not  expecting  overtime" 
to  be  sure  there  were  enough  filters  for  the 
war  effort.  A  tire  dealer  in  Texas  rented  a 
U-haul  to  get  tires  to  the  Army  base  that 
was  transporting  them.  Many  contractors 
went  through  "what-if"  drills  over  and  over, 
and  made  excellent  suggestions  on  speeding 
up  actions.  One  respondent  stated  that  GE’s 
aircraft  engine  division  worked  directly  with 
government  personnel  in  assessing  needs  for 
parts  by  engine  model,  manufacturing 
capacities,  and  the  abilities  of  vendors  to 
provide  raw  materials  and  components. 

Respondents  reported  that  contractors  began 
preparing  and  planning  when  requirements 


became  known,  even  without  contracts  in 
place.  One  respondent  reported  that 
contractor  CEO’s  were  available  for 
discussion  of  any  issue.  Contractor 
employees  at  all  levels  did  not  have  to  be 
asked  to  perform;  they  only  needed 
explanations  of  what  was  required. 
Contracts  were  negotiated  on  the  weekend 
and  contractors  had  people  working  around 
the  clock.  Some  contractors  paid  for 
transportation  upgrades  at  their  own 
expense.  Shipowners  generally  were  very 
responsive  to  the  government’s 
transportation  requirements,  but  because 
they  were  high  in  demand  they  could 
negotiate  at  the  high  end  of  the  price  scale. 
Only  one  respondent  reported  that  some 
contractors  tried  to  take  advantage  of  the 
situation. 

There  was  unanimous  agreement  among  the 
respondents  that  there  was  a  difference  in 
industry  support  during  ODS  as  compared  to 
before  or  after  ODS.  There  was  an  air  of 
cooperation  that  did  not  exist  before  ODS. 
Some  contractors  were  willing  to  work 
without  formal  contracts  and  frequently  at 
no  change  in  contract  price.  "Total  team 
effort"  was  one  respondent’s  description  of 
the  difference  in  industry  support  during 
ODS.  In  one  case  eleven  contractors 
worked  together  as  a  team  to  make  one  ship 
ready. 

Patriotism  was  repeatedly  given  as  the 
reason  for  the  change  in  industry  support 
during  ODS.  Some  other  reasons  were  the 
desire  for  the  company  and  equipment  to 
"look  good,"  and  pride  in  wanting  to 
support  the  systems.  Only  one  respondent 
attributed  the  change  in  the  relationship  with 
industry  to  increased  business  opportunities. 

Waivers  To  the  Regulations.  Many 
suggestions  for  waivers  that  would  have 
been  helpful  during  ODS  were  given.  An 
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increase  to  the  Justification  and  Approval 
(J&A)  review  threshold  for  contracts 
awarded  through  other  than  full  and  open 
competition  would  have  been  helpful  during 
ODS.  Respondents  suggested  waiving  the 
small  business  review  during  ODS.  Other 
waivers  that  would  have  been  helpful  to  the 
respondents  were  concerned  with 
undefinitized  contractual  actions  (UCA’s). 
Respondents  reported  that  the  delegation  of 
approval  for  UCA’s  over  $5  million  to 
lower  levels  would  have  helped  speed  up 
procurements  during  ODS.  Deviations  to 
the  law  restricting  amount  of  expenditures 
which  can  be  made  under  a  UCA  would  also 
have  been  helpful.  Some  other  suggestions 
for  waivers  that  individual  respondents 
reported  would  have  been  helpful  during 
ODS  were  as  follows:  waiver  to  the  Service 
Contract  Act  concerning  wage  determina¬ 
tions,  waiver  of  the  synopsis  publication  and 
response  time,  and  waiver  to  the  necessity 
for  cost  and  pricing  data. 

Several  suggestions  were  made  concerning 
actions  the  individual  military  departments 
could  have  made  during  ODS  to  make 
awarding  of  contracts  faster.  These  are  as 
follows:  the  designation  of  urgent  ODS 
requirements  as  Public  Exigency 
requirements,  the  execution  of  a  class 
"unusual  and  compelling  urgency" 
Justification  and  Approval  (J&A)  for 
certified  ODS  requirements,  the  waiver  of 
all  reviews  except  contracting  officer  and 
Judge  Advocate  General  (JAG)  for  any 
contract  document  implementing  urgent 
ODS  requirements,  the  waiver  of  acquisition 
plan  format,  the  approval  to  automatically 
continue  performance  of  an  ODS  contract  in 
the  face  of  a  protest  at  the  Government 
Accounting  Office  (GAO)  level,  and  the 
delegation  of  Head  of  Contracting  Agency 
(HCA)  individual  FAR  deviation  authority  to 
the  product  division  for  urgent  ODS 
contracts. 


One  respondent  relayed  an  incident 
concerning  waivers  that  occurred  in  the 
early  1980’s  that  was  pertinent  to  ODS.  A 
buyer  in  the  respondent’s  organization  sent 
a  suggestion  to  higher  headquarters 
requesting  waivers  and  changing  the 
delegation  of  authority  to  lower  levels  for 
military  contingency  operations.  The 
headquarters’  response  at  that  time  was  that 
the  suggestion  did  not  need  to  be  processed 
because  waivers  would  be  granted  when 
necessary.  However,  that  organization’s 
request  for  a  class  J&A  during  the  early 
stages  of  ODS  was  not  processed.  The 
headquarters  insisted  that  each  contractual 
action  have  its  own  J&A.  This  respondent 
also  reported  that  later  during  ODS  the 
headquarters  issued  a  request  for 
information  on  suggested  waivers,  changes, 
or  regulatory  relief,  to  include  documented 
support  for  each  suggestion.  A  very  short 
suspense  was  given  to  the  field.  The 
respondent  reported  being  incensed  at  that 
request  because  of  the  headquarters’  lack  of 
response  to  the  class  J&A  request  they  had 
made  months  earlier. 

Several  of  the  respondents  reported 
requesting  a  waiver  or  deviation.  Of  those 
requesting  waivers  or  deviations,  only  two 
reported  that  their  requests  were  granted. 
The  reasons  given  for  not  requesting  a 
waiver  or  deviation  generally  dealt  with  the 
necessity  for  speedy  award  of  the  applicable 
contracts.  One  respondent  reported  that  the 
organization  did  not  request  a  waiver 
because  they  were  told  that  the  request 
would  not  be  approved.  Several  respondents 
reported  that  because  of  the  urgency  of  the 
situation  there  was  not  enough  time  to 
process  a  waiver  request.  Another 
respondent  reported  that  because  the 
expenditure  limit  for  UCA’s  is  a  legal  limit, 
a  change  would  require  a  change  in  the  law, 
which  would  also  have  taken  too  long. 
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Justifications  and  Approvals.  The  dollar 
threshold  for  local  approval  of  Justifications 
and  Approvals  (J&A’s)  during  ODS  varied 
among  the  organizations  surveyed.  Eight 
respondents  reported  that  their  threshold  for 
local  approval  was  $10  million.  Some  other 
thresholds  reported  were  $25  million,  $1 
million,  and  $50  million. 

Approximately  one  third  of  the  respondents 
reported  that  the  majority  of  their 
procurements  which  required  J&A’s  fell 
under  their  dollar  threshold. 

Six  respondents  requested  blanket  J&A’s  for 
ODS.  Only  three  respondents  received 
approval  to  use  a  blanket  J&A  for  ODS. 
One  of  these  reported  that  the  approval  was 
only  for  certain  classes  of  items. 

Those  respondents  not  receiving  approval 
for  a  blanket  J&A  reported  various  reasons. 
One  respondent  reported  that  their  activity's 
headquarters  refused  to  forward  the  request 
to  the  military  department.  Another 
respondent  stated  that  the  organization  was 
advised  by  their  approval  authority  to 
"satisfy  the  requirements  of  the  Competition 
in  Contracting  Act  (CICA)."  [It  is 
interesting  to  note  that  the  use  of  a  blanket 
J&A  under  situations  such  as  ODS  does 
satisfy  the  requirements  of  CICA.] 

Inspections.  Half  of  the  respondents’ 
organizations  have  been  inspected  after 
ODS.  None  of  the  respondents  reported  any 
adverse  findings  related  to  ODS 
procurement  actions.  One  respondent 
replied  that  one  inspection  finding  was  that 
because  the  contracting  staff  was  not 
augmented  to  contend  with  the  increased 
work  load  during  ODS  the  documentation 
processing  was  delayed  or  incomplete. 


originated  much  the  same  as  they  are  in 
peacetime:  from  using  commands  through 
the  program  office  to  the  contracting 
division.  The  requirements  for  food  were 
generated  within  the  contracting 
organization,  based  upon  the  classified  troop 
strength. 

Four  respondents  stated  that  they  received 
requirements  on  a  daily  basis.  Respondents 
reported  requirements  arriving  even  before 
Desert  Shield  began  and  continuing 
throughout  the  conflict  as  requisitions  came 
in,  as  stock  levels  were  inventoried,  as 
studies  were  completed  and  demands 
anticipated. 

In  half  the  cases  requirements  were  clearly 
described.  One  respondent  replied  that  the 
request  was  for  something  to  be  done  to 
make  certain  weapons  better,  not  for  a 
specific  weapon.  Another  respondent  stated 
that  the  requested  delivery  dates  changed 
frequently  causing  changes  to  requests  for 
proposals  (RFP’s).  These  changes  caused 
problems  because  during  ODS  the  normal 
award  time  from  issuance  of  RFP  to  award 
was  three  days  for  that  organization.  One 
respondent  reported  that  the  user  requests 
the  organization  received  were  not  for 
specific  items,  but  were  of  the  "what  if" 
variety.  Another  respondent  stated  that  the 
initial  requirements  were  clear,  but  that  they 
kept  changing  throughout  the  conflict. 

Although  half  the  respondents  replied  that 
funding  was  available  when  the  requirements 
were  first  identified,  lack  of  money  was  a 
problem  for  the  others.  Respondents 
reported  that  frequently  no  funding  was 
available  with  the  requirements.  One  of 
these  reported  using  the  contracting 
organization’s  funding  in  some  cases. 


Requirements.  The  respondents  reported  If  requirements  were  not  clearly  identified 

that  the  requirements  for  ODS  were  or  if  funding  was  not  available  in  a  timely 
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fashion,  the  respondents  performed  a  variety 
of  different  actions  to  assist  in  the  contract 
award  process.  Two  respondents  reported 
that  they  acted  as  if  funds  were  available 
and  found  that  in  most  cases  the  funds 
arrived  before  they  were  ready  to  award  the 
contracts.  One  respondent  replied  that  the 
"contracting  [organization]  was  not  going  to 
be  the  reason  a  contract  award  was  late." 
Others  stated  that  some  efforts  were  delayed 
due  to  the  wait  for  funding. 

Approximately  one  third  of  the  respondents 
reported  that  their  relations  with  the  ultimate 
user  did  not  change  as  a  result  of  the 
experiences  with  ODS.  Others  indicated 
that  a  better  relationship  and  greater 
understanding  of  the  user’s  needs  now 
exists.  Another  respondent  stated  that  better 
communications  links  were  established  with 
the  users  during  ODS  and  have  been 
maintained  to  date.  One  respondent  replied 
that  the  user  still  wants  requirement  ir.ct  as 
quickly  as  they  were  during  ODS. 

Communications.  During  ODS  eight 
organizations  did  not  have  direct 
communications  link-ups  with  individuals  in 
the  area  of  operations.  The  respondents 
who  did  communicate  with  individuals  in  the 
area  of  operations  during  ODS  used  various 
methods:  telephones,  facsimiles,  official 
messages,  electronic  mail,  ship-to-shore 
communication,  and  letters. 

Approximately  half  of  the  respondents 
reported  having  a  dedicated  point  of  contact 
in  the  area  of  operation.  One  of  these  was 
a  person  assigned  there  specifically  for  the 
purpose  of  communicating  with  the  office  in 
CONUS.  The  dedicated  points  of  contact 
were  of  the  following  specialties; 
contracting  specialists,  end  item  users, 
distribution,  logistics  specialists,  and 
engineering. 


The  frequency  of  communication  with  the 
contacts  varied  from  once  a  week  to  more 
than  once  a  day. 

The  communication  with  the  contacts  in  the 
area  of  operations  covered  many  different 
subjects  -  what  was  needed;  when  items 
were  needed;  when  items  would  be 
delivered;  the  redelivery  of  some  items;  the 
marking,  containerization,  or  transport  of 
the  needed  items.  Some  other  subjects  that 
were  discussed  were  leased  vehicles  return 
dates  and  conditions,  problems  encountered 
with  repairing  vessels,  contract 
administrative  items.  requirements 
definition,  backorders,  and  if  contractors 
were  working  on  other  ODS  orders. 

Some  respondents  reported  discussing 
whether  items  would  be  bought  in  the 
CONUS  or  in  i!ie  gulf  area  with  their  points 
of  contact,  'fhe  primary  factors  infiuencing 
the  decision  to  make  purchases  in  the  area 
of  opcr<iiion  or  ('ONlfS  v.cre  quality, 
timeliness,  cost  effectiveness,  and  the 
conformance  of  the  supplies. 

Delivery  Follow-up.  FTilf  the  respondents 
tracked  the  delivery  cd'  the  procured  items  to 
the  area  of  operations  and  reported  a  variety 
of  experiences.  One  respondent  reported  a 
high  success  rate  because  much  of  the 
material  was  hand  carried  by  a  contractor  or 
met  at  both  ends  by  a  contractor  or  military 
person.  Another  reported  that  everything 
was  tracked  from  the  vendor  to  the  port  of 
departure  but  tliat  trace  was  lost  at  the 
tranship  points  in  Spain,  d'his  respondent 
also  noted  that  because  there  was  no  stock 
control  system  in  theater  some  receivers  did 
not  know  vvhetlieror  not  they  received  some 
items.  Another  respondent  reported  that 
transportation  was  a  big  prolilem  because  of 
the  difficulty  in  getting  material  into  the  port 
at  New  Cumberland.  C’ontractors  had 
difficulty  finding  lruvkei>  to  pick  up  their 
material  because  the  truckers  did  not  want 
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their  rigs  tied  up  while  they  waited  to  get 
into  the  port.  Workers  in  the  contractor’s 
plant  who  had  worked  extremely  hard  to 
quickly  manufacture  parts  became  frustrated 
when  they  saw  the  material  sitting  on  the 
dock  awaiting  transportation. 

One  respondent  hired  a  field  service 
representative  from  a  contractor.  Mack 
Truck,  and  another  who  was  familiar  with 
commercial  vehicles.  The  representative 
were  at  the  port  when  the  ships  docked  in 
the  theater  of  operation  so  that  the 
equipment  could  be  successfully  mated  for 
use.  One  respondent  reported  that  vessels 
were  tracked  using  all  types  of 
communication  from  the  time  of  delivery  to 
the  load  port,  ocean  transit  arrival  in 
discharge  port,  until  redelivery  of  vessel  to 
the  ship  owner. 

Contract  Payment.  Over  one  third  of  the 
respondents  stated  that  their  contractors 
encountered  payment  problems  with  ODS 
contracts.  Several  different  reasons  were 
given  for  the  problems.  One  respondent 
attributed  the  problems  to  the  lack  of 
familiarity  of  commercial  contractors  new  to 
dealing  with  DOD.  The  UCA  expenditure 
limit  caused  some  payment  problems 
because  contracts  were  awarded  and  parts 
shipped  before  the  action  could  be 
definitized.  One  respondent  replied  that  part 
of  the  problems  was  due  to  the 
reorganization  of  the  payment  office  at  that 
organization.  Other  reasons  that  were  given 
were  that  the  payment  office  was 
overwhelmed  with  work,  and  a  perception 
that  some  of  the  people  in  the  payment 
offices  did  not  care. 

Workload  Impact.  Only  one  respondent 
answered  that  the  workloads  of  the 
individuals  in  that  organization  did  not  differ 
from  the  normal,  non-ODS  workload.  More 
overtime  was  required  during  ODS  than  for 


the  normal  workload  for  all  except  two 
respondents.  Over  one  third  of  the 
respondents  reported  that  overtime  was  used 
extensively.  They  reported  using  overtime 
constantly.  Another  twenty  percent  used 
overtime  frequently. 

Half  the  respondents  reported  no  use  of 
extra  employees.  The  others  reported  using 
reservists,  temporary  employees,  and 
borrowed  employees  from  other 
organizations.  Retired  civil  service 
annuitants  were  used  during  ODS  in  the 
organizations  of  three  of  the  respondents. 
Employees  in  seventy-five  percent  of  the 
respondents’  organizations  worked  week¬ 
ends  during  ODS.  Many  reported  people 
working  on  holidays  to  support  ODS 
procurement  activities. 

Seventy-five  percent  of  the  respondents’ 
organizations  had  personnel  on-call  via 
beeper  or  restricted  travel  during  off-duty 
hours  during  ODS.  Four  respondents  were 
on-call  constantly. 

Several  additional  comments  were  made 
concerning  the  workload  during  ODS.  One 
respondent  answered  that  routine  buys  were 
postponed  and  individuals  were  not  able  to 
do  as  much  planning  and  follow-up  as  usual. 
ODS  forced  people  to  move  into  more  of  a 
reactive  instead  of  a  proactive  mode. 

A  respondent  replied  that  payment  for 
overtime  was  never  discussed.  One 
respondent  reported  that  productivity 
increased  50%  per  hour  per  person  and  that 
almost  no  sick  leave  was  taken  during  ODS. 
Another  respondent  suggested  that  even 
though  the  DOD  is  downsizing  we  should 
attempt  to  retain  trained,  knowledgeable 
individuals  who  would  be  available  for  use 
in  an  emergency  by  using  part-time,  or  job 
sharing  programs  in  government  offices. 
This  respondent  explained  that  because  it 
takes  three  to  five  years  to  train  buyers, 
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hiring  new  employees  in  an  emergency 
would  not  be  as  effective  as  converting  a 
trained,  part-time  employee  to  full-time 
would  be. 

Sense  of  Urgency.  Individuals  in  the 
contracting  offices  of  most  of  the 
respondents  demonstrated  a  distinct  sense  of 
urgency  in  accomplishing  the  ODS  support 
mission.  Respondents  reported  that  buyers 
and  contracting  officers,  local  review 
committee  members,  legal  staff,  and  price 
analysts  showed  a  sense  of  urgency  during 
ODS. 

Higher  headquarters  personnel  were  reported 
to  demonstrate  a  sense  of  urgency  less  often 
than  the  contracting  office  personnel.  The 
most  common  complaint  was  that  both  local 
and  headquarters  staff  offices  treated  the 
ODS  requirements  as  normal  business.  One 
respondent  answered  that  "those  [people] 
that  were  not  responsive  to  my  needs  could 
be  easily  forced  to  be  so  by  emphasizing 
their  contribution  or  lack  of  same  to  the 


One  third  of  the  respondents  reported  that 
contract  administration  office  personnel 
(Defense  Contract  Management  Command 
(DCMC)  were  responsive  to  the  urgency  of 
the  ODS  situation.  However,  one 
respondent  reported  a  very  frustrating 
episode  in  which  a  DCMAO  transportation 
representative  ordered  a  contractor  not  to 
ship  needed  items  which  consequently 
missed  the  boats  that  the  Program  Executive 
Officer  (PEO)  had  scheduled  specifically. 
The  transportation  contract  had  to  be 
terminated  and  the  contracting  office 
personnel  had  to  work  all  weekend  to  find 
other  trucks  that  could  still  meet  the  boats’ 
departure  dale.  No  reason  was  given  for  the 
representative’s  actions. 

Two  thirds  of  the  respondents  reported  that 


the  contractors  they  worked  with  during 
ODS  demonstrated  a  sense  of  urgency. 

On  the  positive  side,  a  respondent  reported 
that  there  was  not  "one  example  of  someone 
who  refused  to  help.  Some  only  gave  10% 
over  their  usual  amount,  some  gave  much 
more."  One  respondent  answered  that 
"everyone  bent  over  backwards  to  do  their 
jobs"  and  then  asked  what  else  could  be 
done.  Other  respondents  commented  that 
everyone  involved  in  ODS  procurements 
worked  as  a  team  and  went  "above  and 
beyond  the  call." 

Organization  Structure.  Two  thirds  of  the 
respondents  worked  in  functionally 
structured  organizations  before  ODS.  Only 
three  respondents  worked  in  matrixed 
organizations  prior  to  ODS. 

One  third  replied  that  their  organizations 
formed  special  teams  to  support  ODS.  The 
teams  that  were  formed  were  composed  of 
individuals  from  many  different 
backgrounds.  The  respondents  reported  that 
the  makeup  of  the  teams  consisted  of 
representatives  from  the  following  functional 
areas:  comptroller,  logistics,  engineering, 
test  wing,  laboratories,  quality,  technical, 
supply,  procurement,  military  service, 
production,  contracting,  transportation, 
maintenance,  product  assurance,  readiness, 
administrative  contracting  officer  (AGO), 
price  analyst,  supervising  surveyor,  and 
"trade  people." 

The  operation  of  the  teams  differed  from 
organization  to  organization.  One 
respondent  described  the  operation  of  the 
Air  force’s  Rapid  Response  Process  (RRP) 
team.  The  RRP  team  was  set  up  as  a  focal 
point  for  incoming  combat  mission  needs 
statements  (CMNS)  from  ODS  users.  The 
team  was  to  analyze  the  need,  direct  the 
work  to  a  specific  program  office,  and  to 
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present  an  alternate  fix  if  necessary.  The 
team  worked  toward  a  48  hour  turn  around 
from  identified  need  to  a  suggested  fix  and 
the  strategy  to  aecomplisli  tlie  fix.  The  team 
also  acted  as  consultants  to  the  program 
offices  if  needed. 

Several  team  operations  attempted  to  solve 
problems  dealing  with  transportation, 
delivery,  forecasting  amounts,  acceptability 
of  deviations  from  the  specifications, 
modification  of  inspection  criteria,  and 
acceptability  of  substitutes. 

Respondents  reported  that  the  team  members 
were  selected  for  their  high  level  of 
expertise,  'fwo  respondents  answered  that 
the  team  members  were  chosen  by  a  higher 
authority,  fhese  respondents  did  not  give 
information  on  the  basis  of  the  choices. 
Others  replied  iluit  team  members  were 
chosen  who  were  ctimpatible  with  others. 

Only  four  respondents  reitorted  that  the 
teams  were  continued  after  ODS.  One  of 
these  answered  that  the  team  was  continued 
as  the  result  of  the  drawdown  decisions. 
One  negative  rexponse  included  the 
explanation  tfiat  organi/ativins  "cannot  keep 
all  the  best  on  one  team." 

Co-workers  l-xpcriences.  '1  he  majority  of 
respondents  reported  variations  on  the  theme 
of  the  pride  their  co-workers  felt  for  their 
support  of  On.S.  Only  one  respondent 
repcirted  negativ  ely  to  this  t[uestion.  and  that 
response  dealt  wiili  the  lack  of  urgency 
shown  by  individuals  m  tlie  slatf  offices. 

Chidlengfs,  Manv  cli.dlcnges  v\ere  faced 
during  ODS  by  all  ot  the  respondents  to  this 
questionn.iire,  1  he  most  liequent  challenge 
given  by  the  respondents  uas  |)erfornting  .ill 
the  aspects  ol  the  jiroeurement  process  as 
rapidly,  and  acenralclv  .is  possible.  One 
third  of  tlie  respondents  reported  that  one  of 


their  greatest  challenges  was  getting 
individuals  in  staff  and  other  offices  not 
directly  supporting  ODS  to  expedite  ODS 
related  work. 

Another  challenge  was  the  lack  of  a  pre¬ 
planned  streamlined  acquisition  process 
designed  specifically  for  supporting  military 
contingencies.  Respondents  replied  that  the 
recent  personnel  cutbacks  caused  challenges 
in  supporting  ODS  procurements.  Getting 
funds  in  a  timely  manner  was  a  challenge 
during  ODS.  One  respondent  replied  that 
getting  contractors  to  understand  that  the 
war  did  not  mean  that  "anything  goes"  was 
a  great  challenge  during  ODS.  Another 
indicated  a  similar  problem  with  program 
offices  who  attempted  to  use  ODS  priority 
for  non-ODS  requirements.  Another 
reported  the  challenge  of  balancing  ODS 
pr.'"',tics  \vith  the  every  day  program 
prie'i  t.e  order  to  insure  that  both  were 
accomplisaed.  One  of  the  respondents  was 
challenged  with  "getting  the  latest  and  best 
to  the  field  without  getting  ahead  of  spares" 
and  training  requirements.  Two  respondents 
replied  that  overseas  transportation  was  a 
significant  challenge  faced  during  ODS. 

One  respondent  reported  that  the  selection  of 
the  best  contractor  was  a  challenge  during 
ODS  due  to  the  necessity  for  the  decision 
makers  to  consider  the  best  value  in  terms 
of  delivery  schedules,  quality,  as  well  as 
price.  Others  answered  that  finding  new 
procurement  sources  was  a  challenge  during 
ODS.  Anticipating  requirements  was  a 
challenge  during  ODS.  Individuals  in  one 
organization  were  forced  to  move  from  their 
normal  reactive  mode  to  a  more  proactive 
mode  in  order  to  best  serve  the  needs  of  the 
troops  in  the  field. 

Other  respondents  stated  that  dealing  with 
organizations  or  subject  areas  that  were 
rarely  ever  dealt  with  before  (state  level  and 
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DOD  transportation  authorities,  bankers, 
and  payment  offices)  while  still  abiding  by 
contract  terms  created  a  significant  challenge 
during  ODS.  Completing  all  the  required 
contract  file  documentation  under  the  time 
constraint  posed  by  ODS  was  a  major 
challenge.  Another  respondent  replied  that 
obtaining  the  users’  accurate  need  dates 
before  deployment  was  a  challenge  in 
supporting  ODS  procurements.  The 
development  of  “streamlined  internal  office 
procedures  for  managing  fast-paced 
procurements  with  large  numbers  of  offerors 
on  each  solicitation"  was  reported. 

One  respondent  reported  that  getting 
contract  technicians  to  the  theater  and 
obtaining  clearances  to  deploy  were 
significant  challenges  faced  during  ODS. 
Another  respondent  replied  that  assisting 
customers  in  writing  Statements  of  Work 
(SOW)  was  challenging  during  ODS. 
Another  stated  that  prioritizing  multiple 
orders  from  different  sources  for  one 
contractor’s  product  created  a  significant 
challenge  during  ODS. 

Challenges  After  ODS.  The  aftermath  of 
ODS  also  created  challenges  for  the 
respondents  of  this  study.  Half  reported 
challenges  dealing  with  modifying  or 
terminating  contracts  after  ODS  ended.  One 
of  these  respondents  replied  that  modifying 
or  canceling  contracts  without  seeming  to 
destroy  the  industrial  base  or  punish  the 
contractors  proved  to  be  especially 
challenging.  Others  reported  challenges 
encountered  when  making  monetary 
adjustments  to  ODS  contract  modifications 
or  terminations.  The  adjustments  mentioned 
were  the  war  risk  bonus,  fuel  adjustments, 
and  cost  estimates  for  work  completed  at  the 
point  of  "stop  work. " 

Other  respondents  reported  that  a  major 
challenge  they  have  encountered  since  ODS 


ended  was  preparing  for  the  downsizing  of 
DOD  and  being  able  to  support  future 
efforts  with  less  people.  Definitizing  letter 
contracts  was  a  challenge  after  ODS.  One 
of  those  respondents  also  stated  that 
explaining  to  the  program  office  why  we 
could  not  use  UCA’s  and  letter  contracts  on 
a  day-to-day  basis  after  ODS  proved  to  be 
challenging. 

One  respondent  reported  that  understanding 
that  the  organization’s  support  of  ODS  did 
not  end  with  the  official  end  of  the 
hostilities  was  a  challenge.  Another  stated 
that  identifying  excess  levels  of  supplies  in 
theater  created  a  challenge  after  ODS  ended. 
One  respondent  reported  that  returning  stock 
levels  to  that  used  before  ODS  was  a 
challenge.  Getting  equipment  returned  to 
the  CONUS  proved  to  be  challenging  after 
ODS  ended.  The  challenge  of  recognizing 
both  contractors  and  employees  "whose 
outstanding  support  was  obvious"  during 
ODS.  One  respondent  replied  that 
"capturing  the  streamlining  ideas  and 
permanently  improving  the  procurement 
process"  has  been  a  great  challenge  since 
ODS  ended. 

Training.  The  majority  reported  that  they 
did  not  receive  any  special  training  for 
responding  rapidly  to  contracting  needs  for 
a  military  contingency.  Twenty-five  percent 
answered  that  their  training  was  the  years  of 
experience  they  gained  on  the  job.  One 
respondent  replied  that  all  mandatory 
training  requirements  were  received,  but  that 
none  of  the  training  was  designed 
specifically  to  prepare  for  military 
contingencies.  Another  respondent  received 
inter-office  training  on  international  airlift 
purchases. 

One  respondent  stated  that  the  training 
should  have  been  more  structured  and  given 
at  the  beginning  of  ODS. 
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CONCLUSIONS 

Based  upon  the  results  of  the  research,  the 
following  recommendations  are  offered; 

1.  Capture  the  experiences  and  lessons 
learned  from  contracting  support  of  ODS. 
It  may  not  be  the  same  as  future  conflicts, 
but  there  is  much  we  can  learn  from  this 
most  recent  experience. 

2.  Develop  a  "readiness  contacting" 
package  which  identifies  all  the  waivers 
which  can  automatically  be  put  into  place  in 
the  event  of  a  conflict  like  ODS. 
Headquarters  staff  should  take  the  initiative 
and  be  prepared  to  support  contracting 
personnel  who  do  the  acquisitions.  As 
defense  resources  are  decreased,  future 
workload  will  fall  on  even  fewer  shoulders. 
Action  should  be  taken  now  to  provide  relief 
under  any  similar  circumstances  in  the 
future. 

3.  "Wars  start  with  spare  parts."  Identify 
the  critical  spare  parts  for  weapon  systems. 
Recognize  that  there  may  be  different  lists 
for  different  environments  and  conflicts. 
Avoid  JIT  inventory  of  critical  spare  parts. 

4.  Train  contracting  people  on  how  to 
support  a  war.  Include  procurement  in 
wargaming  exercises.  Contracting 
professionals  need  to  be  trained,  primed, 
and  ready  to  react  quickly  when  needed. 

5.  Push  down  decision  making  authority 
and  accountability  where  possible.  We  have 
a  committed,  highly  trained,  and 
increasingly  professional  contracting 
community.  The  authority  for  a  class 
Justification  and  Approval  should  be  granted 
for  future  conflicts. 

6.  Last,  but  most  important  of  all,  recognize 
and  express  appreciation  for  the  committed 


industry  and  government  contracting  people 
who  worked  so  hard  and  made  many 
personal  sacrifices  during  ODS.  With  their 
pride,  spirit,  and  professionalism  they  have 
shown  the  contracting  community  a  pathway 
to  excellence. 
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F-105D  aircraft.  He  later  transitioned  to  the 
F-4  aircraft. 

In  June  1971,  Colonel  Bolton  was  assigned  to 
Ubon  Royal  Thai  Air  Force  Base,  Ubon, 
Thailand,  flying  F-4D/E  for  the  497th  Tactical 
Fighter  Squadron  (Nite  Owls).  While  at 
Ubon,  he  flew  232  combat  missions  -  40  over 
North  Vietnam  —  including  the  first  missions 
after  the  bombing  of  North  Vietnam  was 
resumed.  In  1972,  he  was  assigned  to  Cannon 
Air  Force  Base,  NM,  where  he  flew  the  F- 


II  ID  aircraft  and  served  as  an  instructor  pilot 
and  safety  officer. 

In  1974,  he  was  assigned  to  the  55th  Tactical 
Fighter  Squadron,  Royal  Air  Force,  Upper 
Heyford,  England,  as  an  F-lllE  pilot.  \^ile 
there,  he  served  as  the  squadron  and  wing 
safety  officer,  instructor  pilot,  and  wing  stan¬ 
dardization/evaluation  flight  examiner  and 
scheduler. 

In  1977,  Colonel  Bolton  attended  the  Air 
Force  Test  Pilot  School  at  Edwards  Air  Force 
Base,  CA.  Following  graduation,  he  was 
assigned  to  the  3245th  Test  Wing  at  Eglin  Air 
Force  Base,  FL,  as  a  test  pilot  flying  the  F-4, 
F-111,  and  F-16.  He  also  was  the  F-111  flight 
test  manager  for  the  Armament  Division  there. 
In  1982,  he  was  assigned  to  Aeronautical 
Systems  Division,  Wright-Patterson  Air  Force 
Base,  OH,  as  the  first  program  manager  for 
the  Advanced  Tactical  Fighter  Technologies 
Program,  which  evolved  into  the  F-22  System 
Program  Office. 

In  1986,  the  colonel  was  assigned  to  the  Penta¬ 
gon,  Washington,  D.C.,  first  as  the  F-16  pro¬ 
gram  element  monitor  and  deputy  division 
chief,  Aircraft  Division,  and  later  as  the  divi¬ 
sion  chief.  Low  Observables  Vehicle  Division, 
Office  of  Special  Programs.  He  returned  to 
the  Aeronautical  Systems  Division  as  deputy 
program  director  for  the  B-2  System  Program 
Office  in  August  1988.  In  August  1989,  he 
became  program  director  for  the  Advanced 
Cruise  Missile  System  Program  Office,  Aero¬ 
nautical  Systems  Division,  Air  Force  Systems 
Command.  He  assumed  the  position  of  In¬ 
spector  General,  Headquarters  Air  Force 
Materiel  Command,  Wright-Patterson  Air 
Force  Base,  OH,  in  September  1992. 


VICE  ADMIRAL  WILLIAM  C.  BOWES,  USN 
Commander,  Naval  Air  Systems  Command 


Vice  Admiral  William  Bowes,  a  native  of  New 
York,  graduated  from  the  University  of  Idaho 
and  its  NROTC  Regular  Program  in  June 
1963.  He  is  a  1968  graduate  of  the  United 
States  Naval  Test  Pilot  School  and  received  his 
masters  degree  in  systems  management  from 
the  Naval  Postgraduate  School  in  March  1974. 

Upon  receiving  his  wings  in  December  1964, 
he  joined  the  light  attack  community  at  the 
Naval  Air  Station,  Lemoore,  CA.  Ho  served 
with  VA-113  flying  A-4’s  and  later  with  VA-94 
flying  A-7’s  on  combat  missions  from  the  USS 
KITTY  HAWK  (CV  63),  USS  ENTER¬ 
PRISE  (CVN  65),  and  USS  CORAL  SEA 
(CV  43).  Vice  Admiral  Bowes  flew  350  com¬ 
bat  missions  in  Southeast  Asia  between  1965 
and  1972.  He  served  with  the  VA-195  Dam- 
busters  from  August  1976  until  December 
1979  as  Executive  Officer  and  then  as  Com¬ 
manding  Officer.  During  this  time  he  operat¬ 
ed  with  both  the  Pacific  and  the  Atlantic  fleets 
from  USS  KITTY  HAWK  and  the  USS 
AMERICA  (CV  66). 

An  experienced  test  pilot,  Vice  Admiral  Bowes 
has  had  three  tours  at  the  Naval  Air  Test 
Center,  Patuxent  River,  MD.  He  has  served 
as  an  engineering  test  pilot  in  the  Carrier 
Suitability  Branch,  an  instructor  and  operations 
officer  at  the  Test  Pilot  School,  and  as  the 
Director,  Systems  and  Engineering  Test  Direc¬ 
torate.  Having  piloted  every  je;  aircraft  that 
has  been  in  the  fleet  since  the  1960’s,  he  has 
flown  more  than  5,000  hours  in  over  50  differ¬ 
ent  U.S.  and  foreign  military  aircraft.  He  has 


also  accumulated  900  carrier-arrested  landings 
and  more  than  150  carrier  touch-and-go  land¬ 
ings  during  his  operational  tours  and  during 
carrier  suitability  testing.  He  is  an  associate 
fellow  in  the  Society  of  Experimental  Test 
Pilots. 

Vice  Admiral  Bowes  began  his  program  man¬ 
agement  experience  in  January  1980  in  the 
Naval  Air  Systems  Command,  as  the  F/A-18 
Assistant  Program  Manager  for  Systems  and 
Engineering.  In  December  1983,  he  was 
assigned  as  the  F-14  Aircraft  and  Phoenix 
Missile  System  Program  Manager.  He  man¬ 
aged  the  Phoenix  missile  system  until  June 
1985  and  the  F-14  until  November  1987. 
During  that  tour  of  duty  the  F-14A  (PLUS) 
and  the  F-14D  programs  were  initiated  and 
the  first  production  F-14A  (PLUS)  was  deliv¬ 
ered  to  the  Navy. 

He  was  promoted  to  Rear  Admiral  in  Novem¬ 
ber  1987,  at  which  time  he  was  assigned  as  the 
Director,  Cruise  Missiles  Project.  In  May 
1988,  he  was  given  an  additional  responsibility 
and  became  the  first  director  of  the  Unman¬ 
ned  Aerial  Vehicles  Joint  Project.  In  April 
1990  he  was  designated  the  Program  Executive 
Officer  for  the  Cruise  Missiles  Project  and  the 
Unmanned  Aerial  Vehicles  Joint  Project. 

In  March  1991,  Vice  Admiral  Bowes  became 
the  Commander  of  the  Naval  Air  Systems 
Command. 


MAJOR  GENERAL  JAMES  A.  BRABHAM,  JR.,  USMC 

Commander,  Marine  Corps  Systems  Command 


Major  General  James  A.  Brabham,  Jr.,  is 
currently  serving  as  the  Commander,  Marine 
Corps  Systems  Command,  Quantico,  VA. 

General  Brabham  was  born  August  31, 1939  in 
Upper  Darby,  PA.  He  received  his  commis¬ 
sion  in  June  1962  following  graduation  from 
Cornell  University,  where  he  earned  a  Bache¬ 
lor  of  Civil  Engineering  degree. 

In  December  1962,  after  completing  The  Basic 
School  at  Quantico,  VA,  he  was  assigned  to 
the  1st  Marine  Division,  Camp  Pendleton, 
CA.,  where  he  served  as  a  platoon  leader  in 
the  1st  Bridge  Company,  FMF,  Pacific.  He 
commenced  a  two-year  tour  of  duty  at  the 
Marine  Barracks,  Pearl  Harbor  in  1964,  serv¬ 
ing  as  guard  officer  and  barracks  personnel 
officer.  He  reported  to  the  1st  Shore  Party 
Battalion  in  Chu  Lai,  Vietnam  in  August  1966, 
where  he  served  as  Commanding  Officer  of 
Company  A,  and  subsequently  Commanding 
Officer  of  Headquarters  and  Service  Company. 

From  August  1967  to  July  1968,  he  attended 
the  U.S.  Army  Engineer  School,  Fort  Belvoir, 
VA.  Upon  graduation,  he  was  assigned  to 
Headquarters  Marine  Corps,  Washington, 
D.C.,  for  duty  as  the  Engineer  Equipment 
Programming  Officer  in  the  Office  of  the 
Assistant  Chief  of  Staff  (G-4). 

Ordered  back  to  Vietnam  in  April  1971,  he 
served  as  the  Engineer  Advisor  to  the  Viet¬ 
namese  Marine  Corps,  remaining  in  that 
capacity  through  the  North  Vietnamese  Easter 
offensive  of  April  1972. 

Upon  reassignment  to  Twentynine  Palms,  CA., 
he  assumed  command  of  Provisional  Mainte¬ 
nance  Company,  Force  Troops,  FMF  Pacific. 


He  commanded  that  unit  until  assignment  to 
the  U.S.  Naval  Academy  in  August  1975. 
During  his  tour  at  the  Naval  Academy,  he 
served  as  a  Company  Officer,  Instructor  of 
Leadership,  and  Executive  Assistant  to  the 
Commandant  of  Midshipmen. 

Returning  overseas  in  August  1978,  he  served 
as  Executive  Officer  of  Camp  Garcia,  Vieques, 
Puerto  Rico,  until  December  1978. 

General  Brabham’s  next  assignment  was  as  the 
Program  Coordinator  for  the  Air  Cushion 
Landing  Craft  Development  and  Acquisition 
Program  on  the  staff  of  the  Chief  of  Naval 
Operations,  Washington,  D.C.  On  July  2, 
1981,  he  assumed  command  of  the  2d  Combat 
Engineer  Battalion,  Camp  Lejeune,  NC,  and 
commanded  that  unit  until  July  1983,  when  he 
was  assigned  duties  as  the  Assistant  Chief  of 
Staff,  G-4,  2d  Marine  Division. 

From  August  1984  until  July  1986,  General 
Brabham  was  assigned  duties  as  the  Assistant 
Chief  of  Staff,  G-4,  II  MAF,  Camp  Lejeune, 
NC.  General  Brabham  was  next  assigned  as 
the  Deputy  Director  for  Logistics,  J-4,  USCE- 
NTCOM,  MacDill  AFB,  FL.  While  serving  in 
this  capacity,  he  was  selected  for  promotion  to 
Brigadier  General  in  November  1988.  He  was 
advanced  to  Brigadier  General  on  June  23, 
1989,  and  was  assigned  duty  as  the  Command¬ 
ing  General,  1st  Force  Service  Support  Group, 
FMF,  Camp  Pendleton,  CA,  on  July  14,  1989. 
He  commanded  that  Unit  during  its  participa¬ 
tion  in  Operations  Desert  Shield  and  Desert 
Storm.  On  July  8,  1991,  he  was  assigned  duty 
as  Director,  Marine  Air-Ground  Training  and 
Education  Center.  He  was  advanced  to  his 
present  grade  on  March  10, 1992,  and  assumed 
his  current  assignment  on  June  29,  1992. 


STEPHEN  K.  CONVER 


Vice  President,  Operations  Integration 
Martin  Marietta  Corporation 


On  January  11,  1993,  Stephen  K.  Conver 
joined  Martin  Marietta  Corporation  as  Vice 
President,  Operations  Integration.  Prior  to 
that  time,  he  served  for  three  years  as  the 
Assistant  Secretary  of  the  Army  for  Research, 
Development  and  Acquisition.  In  that  capaci¬ 
ty,  Mr.  Conver  functioned  as  the  Army  Acqui¬ 
sition  Executive,  with  responsibility  for  Re¬ 
search,  Development,  Test  and  Evaluation 
programs;  procurement  programs;  Army  acqui¬ 
sition  policies  and  procedures;  and  worldwide 
security  assistance  programs. 

Mr.  Conver  previously  served  on  the  profes¬ 
sional  staff  of  the  House  Armed  Services 
Committee  as  the  principal  defense  advisor  to 
Congressman  Bill  Dicldnson  (the  Ranking 
Republican)  and  the  21  other  Republican 
Members  of  the  Committee.  Prior  to  joining 
the  HASC  in  1985,  Mr.  Conver  served  for  four 
years  on  the  staff  of  the  Secretary  of  the  Air 
Force  as  Deputy  Assistant  Secretary  (Programs 
and  Budget).  In  this  position,  he  was  responsi¬ 
ble  for  overseeing  the  development,  approval 


and  execution  of  the  Service  budget.  Between 
1975  and  1981,  Mr.  Conver  held  several  posi¬ 
tions  with  the  United  States  Nuclear  Regulato¬ 
ry  Commission,  beginning  as  an  operations 
analyst  and  progressing  to  Chief  of  Analysis 
and  Planning  for  the  Agency. 

After  receiving  his  Bachelor  of  Science  Degree 
from  the  U.S.  Air  Force  Academy  in  1966,  Mr. 
Conver  served  for  nine  years  as  a  commis¬ 
sioned  officer  in  the  Air  Force.  He  earned  a 
Master’s  Degree  in  Operations  Research  from 
Ohio  State  University  in  1970.  Mr.  Conver’s 
military  experience  covered  a  broad  range  of 
subjects,  including  long-range  planning  studies 
in  support  of  advanced  weapons  acquisitions, 
technical  analysis  of  Soviet  weapon  systems, 
and  analysis  of  U.S.  strategic  force  structure. 

Mr.  Conver  was  bom  in  Memphis,  TN  and 
raised  in  Cincinnati,  OH.  He  and  his  wife  Nan 
presently  reside  in  Great  Falls,  VA. 


THOMAS  J.  DOLAN,  JR. 


Holder  of  Acquisition  Law  Chair 
Defense  Systems  Management  College 


Mr.  Dolan  is  the  Acquisition  Law  Chair  at  the 
Defense  Systems  Management  College.  In  this 
position,  he  is  providing  support  to  OSD  and 
the  Congress  in  the  legislative  review  of  the 
Section  800  Panel’s  recommendation. 

Mr.  Dolan  is  a  native  of  Boston,  MA  He 
graduated  from  the  College  of  The  Holy  Cross 
and  received  his  law  degree  from  Suffolk 
University  Law  School.  After  military  service 
in  the  U.S.  Army,  Mr.  Dolan  started  his  civil¬ 
ian  career  with  the  Office  of  Naval  Research 
(ONR),  Boston,  MA,  in  the  administration  of 
research  and  development  contracts/subcon¬ 
tracts  with  universities,  nonprofit,  and  commer¬ 
cial  organizations.  Subsequently,  he  joined  the 
staff  of  the  ONR  Office  at  the  Massachusetts 
Institute  of  Technology  as  a  contracting  officer 
with  administrative  oversight  responsibilities 
for  advanced  research  and  development  pro¬ 
grams  at  the  Draper  Laboratory. 

In  1971,  he  was  assigned  as  the  resident  repre¬ 
sentative  for  ail  DOD  programs  at  Harvard 
University.  In  addition  to  his  business  man¬ 
agement  responsibilities  at  the  University,  Mr. 
Dolan  undertook  a  special  assignment  to 
streamline  procedures  for  performing  contrac¬ 
tor  procurement  system  reviews. 

In  1979,  Mr.  Dolan  transferred  to  ONR  Head¬ 
quarters  to  head  up  the  Acquisition  Plans, 
Policy  and  Evaluation  Branch.  During  this 
assignment,  he  was  appointed  Chairman  of  the 
Noncommercial  Cost  Principles  subcommittee 
of  the  DAR  Council.  In  1983,  Mr.  Dolan  was 
detailed  as  a  special  assistant  to  the  Office  of 
the  Under  Secretary  of  Defense  (Research 
and  Advanced  Technology)  to  provide  support 
on  research  acquisition  policy  matters  with 
special  emphasis  on  DOD’s  Independent 
Research  and  Development  Program. 


Upon  his  appointment  to  the  Senior  Executive 
Service  in  1985,  Mr.  Dolan  became  Director  of 
the  University  Business  Affairs  Directorate  in 
ONR.  In  this  position  he  was  the  senior  line 
manager  of  the  ONR  Field  Contract  Adminis¬ 
tration  Organization  and  the  principal  execu¬ 
tive  for  the  formulation  and  development  of 
effective  business  systems.  During  this  period, 
Mr.  Dolan  organized  and  directed  four  major 
administrative  initiatives-The  Federal  Dem¬ 
onstration  Project,  the  Navy’s  Electronic  Funds 
Transfer  Program,  the  University  Coordinated 
Audit  Program  and  the  Patent  Surveillance 
Program.  These  programs  have  become  oper¬ 
ational  procedures  in  the  field  administration 
of  research  programs. 

In  1991,  he  undertook  a  senior  executive 
assignment  at  the  Defense  Systems  Manage¬ 
ment  College  (DSMC)  to  provide  acquisition 
advice,  counsel  and  support  to  the  DSMC 
Commandant,  Navy  Chair,  and  faculty  and 
students.  During  this  period,  he  provided 
expert  support  to  the  Acquisition  Law  Adviso¬ 
ry  Panel  in  their  efforts  to  streamline  and 
codify  DoD  acquisition  laws. 

Mr.  Dolan  has  been  awarded  the  Presidential 
Rank  of  Meritorious  Executives  in  the  SES, 
the  Navy  Meritorious  Civilian  Service  Award, 
and  the  National  Leadership  Award  from  the 
National  Grants  Management  Association. 

Mr.  Dolan  is  a  member  of  the  Massachusetts 
Bar,  and  a  Certified  Professional  Contracts 
Manager.  He  attended  the  Federal  Executive 
Institute  and  the  Program  for  Senior  Managers 
in  Government  at  the  Kennedy  School  of 
Government.  He  resides  with  his  wife,  Sally, 
in  Fairfax,  VA 


MR.  HIDEAKI  DOMICHI 


Economic  Counselor 
Embassy  of  Japan 


Mr.  Domichi  has  served  as  Economic  Counsel¬ 
or  to  the  Embassy  of  Japan,  Washington,  D.C. 
since  March  1992. 

Mr.  Domichi  previously  held  several  positions 
with  the  Ministry  of  Foreign  Affairs.  His 
positions  included  serving  as  the  Director  for 
the  East  Europe  Division  from  1990  to  1992; 
Director  for  the  Second  African  Division, 
1988-1990;  and  Senior  Advisor  for  the  Policy 
Division  of  the  Economic  Cooperation  Bu¬ 
reau,  1987-1988.  In  addition,  he  was  in  charge 
of  the  Tokyo  Summit  (G7)  for  the  Internation¬ 
al  Press  Division  in  1986-1987, 


Earlier,  Mr.  Domichi  served  as  the  First  Secre¬ 
tary  to  the  Embassy  of  Japan  in  the  United 
Kingdom  and  as  the  First  Secretary  to  the 
Embassy  of  Japan  in  Thailand. 

Mr.  Domichi  also  held  the  following  positions 
during  earlier  employment  with  the  Ministry  of 
Foreign  Affairs:  Deputy  Director  for  the 
Second  Middle  East  Division;  Officer  for  the 
First  North  America  Division;  and  as  Officer 
of  the  Office  of  Law  of  the  Sea  Conference. 

Mr.  Domichi  graduated  from  Cambridge 
University,  United  Kingdom  in  1975  and  from 
Tokyo  University  in  1972. 


CATHLEEN  D.  CARMAN 


Professional  Staff  Member 
Committee  on  Armed  Services 
U.S.  House  of  Representatives 


Ms.  Garman  received  her  MA  in  Administra¬ 
tive  Sciences  (Management  Information  Sys¬ 
tems)  from  George  Washington  University  and 
her  BA  in  Political  Science  and  Journalism 
from  the  University  of  Wisconsin.  Since 
joining  the  House  Armed  Services  Committee 
in  1990,  Ms.  Garman  has  participated  in  the 
development  of  numerous  acquisition  improve¬ 
ment  measures,  such  as  the  provisions  adopted 
as  part  of  the  annual  Defense  Authorization 
Acts  (including  the  Defense  Acquisition  Work¬ 
force  Improvement  Act).  She  has  also  helped 
develop  policies  relating  to  the  Defense  De¬ 
partment  drug  interdiction  program. 


Prior  to  her  present  assignment,  Ms.  Garman 
worked  on  the  House  Small  Business  Commit¬ 
tee  and  was  responsible  for  developing  legisla¬ 
tion  related  to  general  small  business  procure¬ 
ment  and  innovation  (including  minority  busi¬ 
ness  development  reform  and  the  Small  Busi¬ 
ness  Innovation  Research  program). 

Ms.  Garman  began  her  career  on  Capitol  Hill 
as  a  legislative  assistant  for  former  Congress¬ 
man  Nicholas  Mavroules  and  subsequently 
served  as  his  Chief  of  Staff.  Ms.  Garman  also 
has  served  two  years  in  the  Philippines  as  a 
Peace  Corps  Volunteer. 


DAROLD  L.  GRIFFIN 


Principal  Assistant  Deputy  for  Research,  Development  and  Acquisition 

Army  Materiel  Command 


Darold  L.  Griffin  was  bom  on  September  4, 
1931,  in  Rensselaer,  IN.  He  attended  the 
University  of  Cincinnati,  receiving  a  degree  in 
Metallurgical  Engineering  in  1954.  Subse¬ 
quent  to  that,  he  also  completed  a  law  pro¬ 
gram  in  conjunction  with  LaSalle  University. 

In  1955,  after  a  year  in  private  industry,  Mr. 
GrifOn  was  inducted  into  the  Army  and  was 
assigned  to  Picatinny  Arsenal  as  a  ScientiGc 
and  Engineering  Specialist. 

Mr.  Griffin  joined  the  Federal  Service  at 
Picatinny  in  1957  as  an  Engineer  in  the  Indus¬ 
trial  Engineering  Division.  He  has  held  suc¬ 
cessively  more  important  positions  in  conven¬ 
tional  and  nuclear  munitions  resulting  in  his 
selection  as  Assistant  Director  for  Engineering 
and  Production  Base  Modernization  and 
Expansion  in  the  Munitions  Command’s  Pro¬ 
curement  and  Production  Directorate  in  1969. 
In  1970  he  became  the  Deputy  Director  of  the 
Munitions  Command’s  new  Manufacturing 
Technology  Directorate  and  in  January  1973, 
was  selected  as  the  Deputy  Project  Manager 
for  Munitions  Production  Base  Modernization 
and  Expansion. 

In  January  1979,  he  joined  Army  Materiel 
Command  Headquarters  as  Chief  of  the  AMC 


Manufacturing  Technology  Office.  He  became 
the  Associate  Director  for  Systems  Develop¬ 
ment  in  August  1980  and  Assistant  Deputy 
Chief  of  Staff  for  Development,  Engineering, 
and  Acquisition  in  October  1981.  In  July 
1985,  Mr.  Griffin  became  the  Deputy  Chief  of 
Staff  for  Production.  In  December  1989,  he 
became  the  Principal  Assistant  Deputy  for 
Research,  Development  and  Acquisition,  HQ, 
AMC.  He  currently  is  the  Principal  Deputy 
for  Acquisition  for  Headquarters,  AMC,  a 
position  he  has  held  since  August  of  1992. 

Mr.  Griffin’s  activities  and  contributions  to  the 
Department  of  the  Army  have  earned  him 
three  Meritorious  Civilian  Service  Awards, 
Presidential  Rank  of  Meritorious  Senior  Exec¬ 
utive,  a  listing  in  Who’s  Who  in  Finance  and 
Industry,  and  numerous  other  awards  and 
honors.  He  is  active  in  several  professional 
societies  and  organizations  including  the  Ame¬ 
rican  Society  for  Metals,  the  Professional 
Division  of  Alpha  Chi  Sigma  Fraternity,  the 
American  Defense  Preparedness  Association, 
and  is  a  past  member  of  the  Stanhope,  New 
Jersey  Board  of  Education 

Mr.  Griffin  is  married  to  the  former  Arlene  C. 
Papke.  They  have  five  children  and  reside  in 
Fairfax  Station,  Virginia. 


COLONEL  HARRY  H.  HEIMPLE,  USAF 


Deputy  Director  for  Requirements 
Headquarters  Air  Force  Materiel  Command 


Colonel  Harry  H.  Heimple  is  the  Deputy 
Director  for  Requirements,  Headquarters  Air 
Force  Materiel  Command,  Wright-Patterson 
Air  Force  Base,  OH. 

Colonel  Heimple  was  born  September  7, 1945, 
in  Wichita,  KA.  He  graduated  from  Fremont 
High  School  in  Sunnyvale,  CA,  in  1963,  and 
from  Stanford  University  in  1967,  majoring  in 
physics.  He  earned  a  master  of  science  degree 
in  astrophysics  from  Ohio  State  University  in 
1969.  He  was  commissioned  a  second  lieuten¬ 
ant  in  1967;  completed  Squadron  Officer 
School  in  1976;  and  completed  the  French 
Test  Pilot  School  in  1977.  He  was  also  a 
distinguished  graduate  of  the  French  Air  War 
College  and  French  National  War  College  in 
1986. 

After  completing  graduate  school,  Colonel 
Heimple  began  active  duty  in  1969,  attending 
pilot  training  at  Vance  Air  Force  Base,  OK. 
He  then  completed  advanced  fighter  training 
in  F-4’s  at  Davis  Monthan  Air  Force  Base, 
AZ,  and  was  assigned  in  1971  to  the  80th 
Tactical  Fighter  Squadron  at  Kunsan  Air  Base, 
Korea. 

The  colonel  was  assigned  to  the  388th  Tactical 
Fighter  Wing,  Korat  Air  Base,  Thailand,  in 
1972,  and  flew  232  combat  missions  in  F-4E’s, 
112  of  which  were  flown  over  North  Viet- 
nam.Colonel  Heimple  was  assigned  to  the  58th 
Tactical  Fighter  Wing,  Luke  Air  Force  Base, 


..AZ,  in  1973,  as  an  instructor  pilot  in  the  F-4 
and  also  served  on  the  wing  staff. 

Colonel  Heimple  attended  the  French  Test 
Pilot  School  at  Istres,  France,  in  1976,  and 
graduated  number  one  in  his  class  in  July  1977. 
He  returned  to  Edwards  Air  Force  Base  as  an 
instructor  in  the  U.S.  Air  Force  Test  Pilot 
School  for  two  years  and  then  served  as  opera¬ 
tions  officer  for  the  F-16  Combined  Test 
Force.  In  1981,  the  colonel  became  command¬ 
er  of  the  Advanced  Fighter  Technology  Inte¬ 
gration  F-16  Joint  Test  Force,  a  joint 
USAF/NASA  program. 

After  completing  the  French  War  Colleges  in 
March  19^,  the  colonel  was  assigned  to  the 
Directorate  of  Operational  Requirements, 
Office  of  the  Deputy  Chief  of  Staff  for  Re¬ 
search,  Development  and  Acquisition,  Air 
Force  headquarters.  In  May  1987,  Colonel 
Heimple  assumed  command  of  the  Avionics 
and  Special  Programs  Division,  Directorate  of 
Electronic  Combat,  Assistant  Secretary  of  the 
Air  Force  for  Acquisition.  In  September  1988, 
he  became  deputy  program  director  for  the 
Joint  Surveillance  Target  Attack  Radar  System 
Program  Director  (JSTARS)  at  Hanscom  Air 
Force  Base,  MA.  A  year  later,  he  became  the 
System  Program  Director  for  the  Joint  STARS 
program. 

Colonel  Heimple  assumed  his  current  position 
in  January  1993. 


GALEN  I.  HO 


President  and  Chief  Executive  Officer 
IBM  -  Aerospace  Systems  Integration  Corp. 


In  1993,  Mr.  Ho  was  named  IBM-FSC  Vice 
President  and  General  Manager  of  the  Merlin 
Programme  responsible  for  the  entire  project 
and  project  team  in  the  United  Kingdom.  He 
is  also  President  and  Chief  Executive  Officer 
of  the  IBM-Aerospace  Systems  Integration 
Corporation  Subsidiary  responsible  for  aero¬ 
space  business  development  opportunities 
arising  from  this  program  in  Europe  estab¬ 
lished  to  pursue  and  conduct  business  through¬ 
out  the  United  Kingdom  and  Europe.  Mr.  Ho 
is  currently  based  in  IBM,  North  Harbour, 
England. 

In  1988,  Mr.  Ho  was  named  Assistant  General 
Manager,  Programs  at  IBM  Federal  Sector 
Division’s  Owego,  NY  facility.  In  this  position, 
he  has  responsibility  for  ail  Aerospace  Devel¬ 
opment/Production  Programs  including  the 
Navy’s  LAMPS  MK  III;  the  Air  Force’s  Com¬ 
bat  Talon  II,  F-15  Aircraft,  and  NATO  E-3 
Aircraft;  and  the  Army’s  Special  Operations 
Aircraft  (ASOA)  and  Communications  High 
Accuracy  Airborne  Location  System  (CHA- 
ALS).  He  oversees  over  50  programs  with  ap¬ 
proximately  200  active  contracts  as  well  as 
directs  all  the  business  development  activities 
necessary  to  define  strategies  for  future  sales 
in  the  Owego  Site. 

Mr.  Ho  was  named  Assistant  General  Manag¬ 
er,  Systems  Programs  in  September  1986.  His 
responsibilities  included  avionics  systems 
development  for  programs  such  as  LAMPS 
MK  III,  Combat  Talon  II,  HH-60A,  LXH/- 
ARTI,  V-22  Avionics  Subsystem,  and  MH-531. 
He  also  managed  the  System  Engineering 
Practices  Development  for  the  Owego  facility. 


In  1984,  Mr.  Ho  was  promoted  to  Program 
Director,  Processor  Systems  Programs  with  res¬ 
ponsibility  for  directing  all  new  avionics  com¬ 
puter  development  programs  and  managing  all 
on-going  avionics  computer  programs  from 
development  through  production  phases. 

The  1981,  he  was  named  Assistant  to  the  IBM- 
FSD  President,  responsible  for  Technical  and 
Administrative  Management  within  the  Presi¬ 
dent’s  office. 

From  1974  to  1981,  he  held  various  manage¬ 
ment  jobs  which  included  responsibility  for  the 
Radar  Weapon  Data  Link  Program,  Joint 
Tactical  Information  Distribution  System 
(JTIDS)  Expendable  Terminal  Development, 
Product  Definition  of  Military  Processor 
Systems,  and  Business  Area  Strategies  for  both 
FSD  Owego’s  Processor  Systems  and  Commu¬ 
nication  Systems  Programs. 

Mr.  Ho  joined  IBM  in  1972  as  a  Staff  Engi¬ 
neer  in  Tactical  Avionics  Systems  at  IBM’s 
Federal  Systems  Division  plant  in  Owego.  He 
was  appointed  Manager  of  Advanced  Naviga¬ 
tion  and  Strike  Systems  at  Owego  in  1974. 

Prior  to  Mr.  Ho’s  employment  at  IBM,  he  was 
an  officer  in  the  U.S.  Air  Force  assigned  to 
the  Aeronautical  Systems  Division  responsible 
for  Processor  Subsystems  Development,  He 
attained  the  rank  of  Captain  and  holds  awards 
for  Vietnam  Combat  Service  and  the  Meritori¬ 
ous  Service  Medal. 

Mr.  Ho  received  a  BSEE  from  Oregon  State 
and  did  graduate  studies  at  Ohio  State. 


GERALD  J.  LOHAN 


Head  of  Trade  Section 

Delegation  of  the  Commission  of  the  European  Communities 


Gerhard  Lohan  is  Counselor  for  trade  and 
commercial  affairs  at  the  Delegation  of  the 
Commission  of  the  European  Communities. 
He  took  up  this  post  on  1  September  1992. 


More  recently,  he  was  in  charge  of  coordina¬ 
tion  of  DG  Ill’s  position  on  competition 
policies,  with  particular  emphasis  on  anti-trust 
matters  and  merger  control. 


Before,  he  worked  for  13  years  for  the  Com¬ 
mission  in  its  Brussels  headquarters,  of  which 
three  years  were  spent  in  the  Directorate 
General  for  External  Relations  (DG  I)  and  ten 
years  in  the  Directorate  General  for  Internal 
Market  and  Industrial  Affairs  (DG  III). 

His  job  experience  includes  trade  policy  of  the 
European  Communities  (EC)  with  Japan,  and 
responsibilities  in  the  field  of  motor  vehicle 
industry  policy.  In  the  context  of  the  "Internal 
Market  -  1992"  objective,  Mr.  Lohan  was 
closely  involved  in  the  legislative  program 
concerning  public  procurement. 


In  the  Washington  Delegation,  Mr.  Lohan’s 
responsibilities  comprise  ECAJS  bilateral  trade 
and  commercial  policy  affairs,  as  well  as  multi¬ 
lateral  matters. 

Mr.  Lohan  has  studied  law  in  Germany  and 
worked  as  manager  of  a  foreign  trade  associa¬ 
tion  before  joining  the  EC  Commission.  He 
resides  in  Washington  with  his  wife  Kristin  and 
their  three  boys,  Make,  Felix  and  Jesco. 


JAMES  A.  McDIYITT 


Senior  Vice  President 
Government  Operations  &  International 
Rockwell  International  Corporation 


James  A.  McDivitt  was  appointed  senior  vice 
president,  government  operations  and  interna¬ 
tional  for  Rockwell  International  Corporation 
in  November  1990.  He  is  a  corporate  officer 
and  a  member  of  the  company’s  management 
committee. 

Mr.  McDivitt  has  held  a  series  of  executive 
positions  with  Rockwell  including:  senior  vice 
president,  Government  Operations,  executive 
vice  president.  Electronic  Operations;  execu¬ 
tive  vice  president.  Defense  Electronics  Oper¬ 
ations;  senior  vice  president  of  science  and 
technology,  and  senior  vice  president  for 
strategic  management.  He  joined  Rockwell  in 
January  1981. 

Prior  to  beginning  his  Rockwell  career,  Mr. 
McDivitt  was  executive  vice  president  and  a 
member  of  the  board  of  directors  of  Pullman 
Inc.,  in  Chicago.  He  joined  that  company  in 
1975. 

Mr.  McDivitt  earlier  was  executive  vice  presi¬ 
dent  and  a  director  of  Consumers  Power  Co. 
of  Jackson,  MI. 

He  entered  the  U.S.  Air  Force  in  1951  and 
was  commissioned  in  May  1952.  He  graduated 
from  the  U.S.Air  Force  Experimental  Test 
Pilot  School  in  March  1960  and  the  U.S.  Air 
Force  Aerospace  Research  Pilot  Course  in 
1961. 

In  1962,  Mr.  McDivitt  was  assigned  to  the 
National  Aeronautics  and  Space  Administra¬ 
tion  (NASA)  as  an  astronaut.  He  was  the 


command  pilot  for  Gemini  IV,  the  66-orbit, 
four-day  space  mission  in  1965;  and  command¬ 
ed  the  Apollo  9  earth  orbital  flight  in  1969. 
He  established  six  world  records  for  space 
flight.  Mr.  McDivitt  was  the  Apollo  Space¬ 
craft  Program  Manager  from  1%9  to  1972, 
with  overall  management  responsibility  for 
Apollo  12  through  16.  He  retired  from  the 
Air  Force  in  1972  as  a  brigadier  general. 

Earlier  in  his  military  career,  Mr.  McDivitt 
flew  145  combat  missions  during  the  Korean 
War.  His  service  decorations  include  two 
distinguished  service  medals,  four  distinguished 
flying  crosses,  five  air  medals  and  the  Republic 
of  Korea  Chung  Moo  medal. 

Born  June  10,  1929,  in  Chicago,  Mr.  McDivitt 
received  a  bachelor  of  science  degree  in  aero¬ 
nautical  engineering  in  1959,  and  honorary 
doctor  of  astronautical  science  in  1965  from 
the  University  of  Michigan.  He  was  awarded 
honorary  doctor  of  science  degrees  by  Seton 
Hall  University  in  1969  and  Miami  University 
in  1970,  and  an  honorary  doctor  of  laws  by 
Eastern  Michigan  University  in  1975. 

Mr.  McDivitt  serves  on  the  advisory  council  of 
the  University  of  Michigan  College  of  Engi¬ 
neering.  He  is  also  the  charter  national  chair¬ 
man  of  the  annual  fund  for  the  College. 

Mr.  McDivitt  is  a  fellow  of  the  Society  of 
Experimental  Test  Pilots  and  the  American 
Aeronautical  Society.  He  is  also  a  member  of 
the  American  Institute  of  Aeronautics  and 
Astronautics. 


LAURA  BETH  SHERMAN 


Assistant  General  Counsel 
U.S.  Trade  Representative 
Department  of  State 


Ms.  Sherman  assumed  the  position  of  Assistant 
General  Counsel,  U.S.  Trade  Representative, 
Department  of  State  in  September  1992. 
Previously,  she  served  as  attorney-advisor  to 
the  Office  of  Ethics  and  Personnel,  Office  of 
the  Legal  Adviser,  Department  of  State.  In 
this  position,  she  was  the  alternate  designated 
agency  ethics  official,  providing  advice  to 
department  principals  and  all  other  employees 
on  ethics  and  conflict  of  interest  statutes  and 
regulations.  Hatch  Act  questions,  nepotism, 
and  other  personnel  issues. 

Earlier,  she  served  as  Associate  (Corporate)  at 
Paul,  Weiss,  Rifkind,  Wharton  &  Garrison  in 
New  York  (1985  to  1990).  Here,  she  was  re¬ 
sponsible  for  the  preparation  and  negotiation 
of  financing  documents,  acquisition  agree¬ 
ments,  joint  venture  contracts  and  licensing, 
technology  transfer,  employment  and  other 
commercial  contracts;  preparation  of  securities 
filings;  advice  to  clients  on  general  corporate 
matters;  and  pro  bono  activities  for  the  Law¬ 
yers  Committee  for  Human  Rights  and  Helsin¬ 
ki  Watch. 

From  1982  to  1985,  Ms.  Sherman  was  a  law 
clerk  in  the  Office  of  International  Claims  and 
Investment  Disputes,  Office  of  the  Legal 
Advisor,  Department  of  State.  She  prepared 
Statements  of  Defense  for  claims  filed  against 
the  U.S.  to  Iran-U.S.  Claims  Tribunal;  pre¬ 
pared  memorials  on  questions  of  interpretation 
of  the  Algiers  Accords;  and  analyzed  merits 
and  recommendations  for  resolution  of  U.S. 


citizen  claims  against  Egypt,  Ethiopia,  Roma¬ 
nia,  and  other  countries. 

As  country  director  for  Papua,  New  Guinea, 
Foundation  for  the  Peoples  of  the  South 
Pacific,  Port  Moresby,  Papua,  New  Guinea 
(1979  to  1981),  Ms.  Sherman  was  responsible 
for  the  administration  of  economic  and  social 
development  projects  with  a  total  budget  of 
$300,000  per  year;  proposal  writing,  evaluation 
and  monitoring  of  projects;  and  providing 
technical  services  to  project  holders. 

Ms.  Sherman  was  a  professional  staff  member 
of  the  Select  Committee  on  Narcotics  Abuse 
and  Control,  House  of  Representatives  in 
1979,  where  she  coordinated  Committee 
oversight  hearings;  performed  legislative  re¬ 
search  and  drafting;  and  drafted  speeches  and 
reports.  Prior  to  that,  she  was  administrative 
assistant  to  the  Director,  Institute  of  Interna¬ 
tional  Relations,  translating  (Chinese  to  Eng¬ 
lish)  and  editing  articles  for  the  Institutes’ 
journal  and  organizing  international  conferenc¬ 
es. 

Ms.  Sherman  received  a  J.D.  degree  (magna 
cum  laude)  in  1985  from  Georgetown  Univer¬ 
sity  Law  Center;  a  Ph.D  in  1978  from  the 
University  of  Sofia,  Bulgaria;  and  a  B.A  (cum 
laude)  in  1973  from  Cornell  University  with 
majors  in  Chinese  studies  and  economics.  She 
has  also  authored  numerous  publications. 


PATRICK  D.  SULLIVAN 


Assistant  Vice  President 
Aerospace  Industries  Association  of  America 


As  Assistant  Vice  President,  Aerospace  Pro¬ 
curement  Service,  Aerospace  Industries  Asso¬ 
ciation  of  America  (AIA),  Mr.  Sullivan  special¬ 
izes  in  the  areas  of  procurement,  finance  and 
contract  management.  His  professional  in¬ 
volvement  in  government  contracting  spans 
over  25  years. 

Since  joining  AIA,  Mr.  Sullivan  has  been  the 
association’s  project  officer  for  a  number  of 
important  legislative  and  regulatory  issues 
affecting  the  aerospace  industry,  including 
Work  Measurement,  Independent  Research 
and  Development  (IR&D),  Material  Manage¬ 
ment  and  Accounting  Systems  (MMAS),  Drug 
Free  Workforce/Workplace  and  Total  Quality 
Management  (TQM).  He  is  the  professional 
staff  officer  responsible  for  the  AIA  Legal 
Committee  and  the  Facilities  and  Property 
Committee. 

Prior  to  joining  AIA,  Mr.  Sullivan  was  a  Se¬ 
nior  Associate  with  Harbridge  House,  Inc., 
where  he  provided  consulting  services,  pre¬ 
pared  instructional  materials  and  presented 
programs  on  acquisition  management  for  major 
defense  contractors. 

Prior  to  his  association  with  Harbridge  House, 
Mr.  Sullivan  served  for  four  years  as  the  Man¬ 
ager  of  Contracts  and  Long-Range  Planning 
for  Data-Design  Laboratories.  Earlier,  Mr. 
Sullivan  was  a  Vice  President  with  Manage¬ 
ment  Concepts,  Inc.  In  this  capacity,  he 
taught  procurement  courses  to  employees  of 
various  civilian  agencies. 

Mr.  Sullivan,  who  retired  from  the  Navy  with 
the  rank  of  Captain,  served  in  several  staff 
positions  involving  the  acquisition  of  major 
defense  systems.  Prior  to  his  retirement,  he 
was  the  Director  of  Weapons  System  Acquisi¬ 


tion  in  the  Office  of  the  Deputy  Under  Secre¬ 
tary  of  Defense,  Acquisition  Policy,  DDR«feE. 

Before  being  assigned  to  OSD,  Mr.  Sullivan 
served  as  Special  Assistant  for  Weapons  Sys¬ 
tem  Acquisition  to  the  Deputy  Commander  of 
Contracts  in  the  Naval  Electronic  Systems 
Command.  He  was  also  the  Contracting 
Officer  for  the  MARISAT  program,  which  was 
the  first  government  lease  of  a  commercial 
communications  satellite. 

Other  Navy  Assignments  included  serving  as 
Chief,  Operations  Branch,  Defense  Contract 
Administration  Services  (DCAS)  Headquar¬ 
ters.  While  at  DCAS,  he  was  loaned  for  a 
year  to  the  congressional  Commission  on 
Government  Procurement,  where  he  was  a 
member  of  the  Reports  and  Management 
Control  Systems  study  group. 

Mr.  Sullivan  has  held  the  position  of  Contracts 
and  Materials  Officer  in  the  office  of  the 
Supervisor  of  Shipbuilding,  Overhaul  and 
Repair  at  Bath  Iron  Works. 

Mr.  Sullivan  has  been  an  adjunct  professor 
with  the  American  University  Kogod  School  of 
Business  Administration  where  he  taught 
Major  Systems  Acquisitions.  He  is  a  Certified 
Professional  Contracts  Manager,  a  Fellow, 
National  Contract  Management  Association, 
and  a  member  of  the  National  Property  Man¬ 
agement  Association. 

Mr.  Sullivan  holds  a  B.S.  degree  in  education 
from  Miami  University  and  a  Masters  degree 
in  business  administration  from  Harvard  Busi¬ 
ness  School. 


NICHOLAS  M.  TORELLI,  JR. 
Consultant 


Nicholas  M.  Torelli,  Jr.  served  as  the  Deputy 
Assistant  Secretary  of  Defense  for  Production 
Resources  from  November  1990  until  February 
1993.  He  was  responsible  for  policy  develop¬ 
ment,  implementation  and  oversight  of  produc¬ 
tion  resources  for  weapons  systems,  including 
production  readiness,  quality,  manufacturing 
modernization,  specifications  and  standards, 
and  the  industrial  base.  Mr.  Torelli  was  the 
principal  architect  in  formulating  the  DoD 
acquisition  strategy  in  its  relationship  to  the 
capabilities  of  the  industrial  base  supporting 
national  defense. 

Mr.  Torelli  graduated  from  the  U.S.  Naval 
Academy  and  the  Navy  Nuclear  Power  Train¬ 
ing  Curriculum  before  serving  in  the  Navy 
aboard  Attack  Submarines.  He  then  spent  12 
years  in  the  defense  industry  in  leadership 
positions  in  systems  engineering,  production 
management  and  program  management  provid¬ 
ing  tactical  missiles  and  combat  systems  to  the 
Army,  Navy,  and  Air  Force. 


In  his  most  recent  industry  position,  Mr.  Tor¬ 
elli  played  a  major  role  in  the  turnaround  of  a 
weapons  systems  manufacturing  division.  He 
was  instrumental  in  developing  a  culture  of 
"Quality  First"  within  the  division  workforce, 
establishing  guidelines  for  transition  of  pro¬ 
grams  from  development  into  production, 
helping  to  establish  a  "gainsharing"  program 
for  employee  involvement  in  division  success, 
and  leading  an  organizational  development 
team  in  overhauling  the  division’s  procedural 
system. 

Mr.  Torelli  is  currently  employed  as  a  private 
consultant  to  industry,  providing  services  in 
industrial  base  policy,  strategic  planning,  quali¬ 
ty,  and  the  integration  of  products  and  pro¬ 
cesses  for  systems  development. 


REAR  ADMIRAL  LEONARD  VINCENT,  SC,  USN 


Deputy  Director  -  Acquisition 
Commander,  Defense  Contract  Management  Command 


Since  December  1992,  in  his  role  as  deputy 
director  for  acquisition  management.  Admiral 
Vincent  has  been  responsible  for  oversight  of 
DLA’s  $12  billion  procurement  operations 
executed  by  3,800  procurement  personnel 
located  throughout  the  Agency  and  for  its 
19,000-person  contract  administration  organi¬ 
zation.  Defense  Contract  Management  Com¬ 
mand,  initiated  to  streamline  contract  manage¬ 
ment,  consolidates  virtually  all  defense  contract 
administration  worldwide,  and  administers 
contracts,  accepts  quality  products  and  pays 
the  contractors  on  behalf  of  all  soldiers,  sail¬ 
ors,  airmen  and  Marines. 

Rear  Admiral  Vincent  entered  the  Naval 
Reserve  program  as  a  seaman  recruit  in  Octo¬ 
ber  1961.  Upon  graduation  from  Southeastern 
State  Teachers  College  in  Oklahoma,  he 
received  a  commission  from  the  Officers 
Candidate  School,  Newport,  RI,  in  July  1965 
as  an  ensign  in  the  Supply  Corps,  U.S.  Navy. 

Following  commissioning,  he  attended  the 
Navy  Supply  Corps  School  in  Athens,  GA 
His  first  duty  station  was  at  the  Naval  Ammu¬ 
nition  Depot,  Hawthorne,  NV,  where  he 
served  as  the  disbursing  and  accounting  officer 
until  July  1968.  His  next  two  years  were  spent 
out  of  Naval  service.  Vincent  was  an  accoun¬ 
tant  for  North  American  Rockwell  and  an 
account  manager  for  Investors  Diversified 
Services,  both  in  Tulsa,  OK. 

Admiral  Vincent  returned  to  the  Navy  in  1970 
and  was  assigned  as  a  supply  officer  with  the 
Naval  Special  Warfare  Group  in  Little  Creek, 
VA  In  October  1971,  he  was  assigned  as  the 
first  supply  officer  in  the  newly  established 
Naval  Inshore  Warfare  Command,  Atlantic. 


In  October  1972,  he  reported  to  the  USS 
PENSACOLA  as  a  supply  officer.  Two  years 
later,  he  attended  the  Armed  Forces  Staff 
College,  Norfolk,  VA  He  then  earned  a 
master’s  in  business  administration  from 
George  Washington  University  in  August  1976. 

Upon  completion  of  his  degree.  Admiral 
Vincent  served  as  director  of  the  contracting 
department  at  the  Naval  Supply  Center  Puget 
Sound  until  July  1979.  From  August  1979  to 
June  1982,  he  was  a  contracting  officer  for  the 
FFG  construction  program  and  surface  ship 
overhauls  as  the  Supervisor  of  Shipbuilding 
and  Repair,  Bath,  ME. 

He  returned  to  sea  duty  in  July  1982  as  supply 
officer  in  the  submarine  tender  USS  Dixon 
(AS-37),  with  additional  duty  as  staff  supply 
officer  for  the  commander,  Submarine  Squad¬ 
ron  Three.  In  August  1984,  he  became  the 
director  of  the  combat  systems  department  of 
the  Navy  Ships  Parts  Control  Center,  Mechan- 
icsburg,  PA  In  August  1986,  he  was  reas¬ 
signed  as  the  director  of  contracting. 

In  July  1989,  Admiral  Vincent  assumed  com¬ 
mand  of  the  then  Defense  Contract  Adminis¬ 
tration  Services  Region  Los  Angeles,  an  activi¬ 
ty  of  DLA  From  July  until  October  1990,  he 
was  the  special  assistant  to  the  commander  of 
the  Defense  Contract  Management  Command 
in  Aexandria,  VA  In  1990,  he  became  com¬ 
mander  of  the  Defense  Contract  Management 
Command  International  in  Dayton,  OH,  and 
from  September  1991  to  November  1992,  he 
was  assigned  as  the  assistant  commander  for 
contracts  at  the  Naval  Ar  Systems  Command, 
Washington,  D.C. 


DEBORAH  L.  WINCE-SMITH 


Senior  Fellow 

Council  on  Competitiveness 


Deborah  L.  Wince-Smith,  the  first  assistant 
secretary  for  technology  policy  in  the  U.S. 
Department  of  Commerce  under  President 
George  Bush,  has  joined  the  Council  on  Com¬ 
petitiveness  as  a  senior  fellow.  The  Council 
on  Competitiveness  is  a  private,  non-profit 
coalition  of  chief  executives  from  leading 
businesses,  organized  labor,  and  academia 
whose  mission  is  to  improve  the  competitive¬ 
ness  of  U.S.  industry  and  its  workers  in  an 
international  marketplace  in  a  way  that  builds 
a  rising  standard  of  living  at  home. 

Assistant  Secretary  Wince-Smith  served  as  the 
principal  officer  in  the  Commerce  Department 
for  developing,  coordinating,  and  advocating 
policies  to  increase  the  role  of  technology  in 
enhancing  the  competitiveness  and  economic 
well-being  of  the  United  States.  She  led  three 
operating  units  which  constitute  the  Office  of 
the  Assistant  Secretary  for  Technology  Policy 
(OASTP),  an  agency  within  the  Technology 
Administration.  They  are:  the  Office  of 
Technology  Policy  Analyses  and  Studies,  the 
Office  of  International  Technology  Policy  and 
Programs,  and  the  Office  of  Technology  Com¬ 
mercialization.  These  offices  act  as  catalysts 
and  help  to  build  new  and  innovative  alliances 
among  industry,  government,  and  academia  to 
meet  the  challenge  of  developing  new  technol¬ 
ogies  and  bringing  them  to  the  global  market¬ 
place. 

OASTP  complements  U.S.  industry’s  efforts  to 
deploy  technology  by  advocating  Federal 
policies  and  programs  that  support  industry’s 
ability  to  commercialize  technology.  It  also 
works  with  U.S.  industry  to  encourage  the 
commercial  use  of  Federal  R&D,  and  the 
adoption  of  modern  manufacturing  technolo¬ 
gies  and  methods.  A  new  and  innovative 
program,  the  Strategic  Partnerships  Initiative, 
promotes  multi-firm  and  inter-industry  cooper¬ 


ation  to  reduce  cost  and  risk  in  the  develop¬ 
ment  and  commercialization  of  enabling  tech¬ 
nologies. 

OASTP  advocates  international  science  and 
technology  policies  that  promote  U.S.  competi¬ 
tiveness.  Key  responsibilities  include  increas¬ 
ing  industry’s  access  to  foreign  technologies 
and  protecting  U.S.  intellectual  property  rights 
and  commercial  interests  in  international  R&D 
cooperation.  OASTP  also  coordinates  the 
U.S.  response  to  proposed  international  re¬ 
search  programs  such  as  the  Intelligent  Manu¬ 
facturing  Systems  initiative  put  forth  by  Japan. 

Prior  to  joining  the  Technology  Administra¬ 
tion,  Ms.  Wince-Smith  served  from  1984-1989 
as  Assistant  Director  for  International  Affairs 
and  Global  Competitiveness  in  the  White 
House  Office  of  Science  and  Technology 
Policy.  She  was  actively  involved  in  developing 
and  implementing  Presidential  international 
science  and  technology  policies  and  programs, 
and  the  design  of  Presidential  initiatives  in 
superconductivity,  technological  competitive¬ 
ness,  and  Head  of  Government  Science  and 
Technology  Agreements.  As  a  National  Sci¬ 
ence  Foundation  program  manager  from  1976- 
1984,  she  was  responsible  for  managing  bilater¬ 
al  research  activities  between  U.S.  scientists 
and  engineers  and  their  counterparts  in  East¬ 
ern  Europe. 

Ms.  Wince-Smith  received  her  Master’s  De¬ 
gree  from  Cambridge  University  in  1974,  and 
graduated  Phi  Beta  Kappa  and  Magna  Cum 
Laude  from  Vassar  College  in  1972. 

A  native  of  Akron,  OH,  Ms.  Wince-Smith 
currently  resides  in  McLean,  VA  with  her 
husband.  Ambassador  Michael  B.  Smith,  and 
their  two  sons. 
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